The free-scoring Premier League side found themselves 2-0 behind at half-time and staring elimination in the face, however City came roaring back to seal victory, with Sergio Aguero grabbing a hat-trick.
Fernando Forestieri and Troy Deeney had shocked the Etihad with two goals in nine minutes, and the Championship side held the lead until the hour mark.
Aguero grabbed the first goal back, and then levelled with 11 minutes to play, before Aleksandar Kolarov put City ahead for the first time in the 87th minute.
The win was confirmed by Aguero's 25th goal of the season in all competitions in the third minute of stoppage time.
Manuel Pellegrini's side were one of eight top-flight clubs to win their fourth-round ties, but Crystal Palace were knocked out by holders Wigan Athletic.
Uwe Rosler's side went ahead through Ben Watson in the 36th minute, and although Aaron Wilbraham levelled for Palace, it was the Championship side who progressed thanks to James McClean's first goal for the club.
Liverpool, Everton, Southampton and Hull City also had no problem in making their way into the next round with comfortable wins.
Victor Moses and Daniel Sturridge helped Liverpool to beat Bournemouth 2-0, while the same scoreline was enough for Hull to see off Southend United - managed by former Hull boss Phil Brown - with Matty Fryatt grabbing both goals at Roots Hall.
Guly do Prado's first goal in more than two years put Southampton ahead against Yeovil Town, before Sam Gallagher's first senior strike sealed a 2-0 win at St Mary's.
Steven Naismith struck twice in Everton's 4-0 win over Stevenage, with John Heitinga and Magaye Gueye completing the win, although the victory was overshadowed by a double leg break suffered by Bryan Oviedo.
Charis Mavrias' first Sunderland goal ended non-league Kidderminster's brave cup run 1-0, while substitute Fraizer Campbell struck five minutes after coming on to send Cardiff City through against Bolton Wanderers by the same scoreline.
Two goals in as many minutes from Wilfried Bony saw Swansea City come from behind to beat Birmingham City 2-1, while Charlton Athletic, Brighton and Hove Albion and Sheffield Wednesday all progressed - seeing off Huddersfield Town, Port Vale and Rochdale respectively.
