Newport's win over Leicester, and Oldhams defeat of Fulham is the first time two top-flight sides have been knocked out of the FA Cup by teams from the fourth tier or below on the same day since 7 January 1989.
The penalty came just three minutes after Rachid Ghezzal had levelled for Leicester, who had trailed since Newport's well-worked opener, which was headed in by Jamille Matt from Robbie Willmott's cross after just 10 minutes.
Puel was sharply criticised for naming an under-strength side for the Carabao Cup quarter-final defeat by makeshift Manchester City opponents last month.
Phil Foden, the England Under-21 worldwide, got on the end of a cross from Ilkay Gundogan in the 43rd minute to score his first home goal for City before Rotherham defender Semi Ajayi turned the ball into his own net.
Claudio Ranieri's Fulham reign has got off to a hard start and it got worse for the former Leicester boss at the hands of Pete Wild's Oldham.
There seemed little chance of an upset alongside the River Thames when Denis Odoi gave Fulham the lead in the second half but Sam Surridge equalised from the penalty spot.
"I think we had a good team with eight who played winning the title with Leicester, and a lot of experienced players and quality on the pitch", Puel, who made seven changes from the side that beat Everton on New Year's Day, told reporters.
Striker Shaquile Coulthirst scored a 21st-minute penalty to send Barnet through to the fourth round, where they'll be the only non-league team to feature in the round of 32.
"It's unbelievable", Barnet captain Callum Reynolds said after the north London club's triumph.
"They made a fantastic upset, it will be a fantastic memory for them".
QPR defeated Championship leaders Leeds by the same scoreline, Jake Bidwell grabbing a 75th-minute victor for the west London club.
Leicester have never won the FA Cup having been runners-up four times, the last in 1969.
Elsewhere, in an all-Premier League clash, Brighton won 3-1 away to south coast rivals Bournemouth.
Manchester City swept to its biggest win under Pep Guardiola with a 7-0 thrashing of second-tier Rotherham in the third round of the FA Cup on Sunday.