The free-scoring Premier League side found themselves 2-0 behind at half-time and staring elimination in the face, however City came roaring back to seal victory, with Sergio Aguero grabbing a hat-trick.
Fernando Forestieri and Troy Deeney had shocked the Etihad with two goals in nine minutes, and the Championship side held the lead until the hour mark.
The win was confirmed by Aguero's 25th goal of the season in all competitions in the third minute of stoppage time.
Manuel Pellegrini's side were one of eight top-flight clubs to win their fourth-round ties, but Crystal Palace were knocked out by holders Wigan Athletic.
Uwe Rosler's side went ahead through Ben Watson in the 36th minute, and although Aaron Wilbraham levelled for Palace, it was the Championship side who progressed thanks to James McClean's first goal for the club.
Guly do Prado's first goal in more than two years put Southampton ahead against Yeovil Town, before Sam Gallagher's first senior strike sealed a 2-0 win at St Mary's.
Steven Naismith struck twice in Everton's 4-0 win over Stevenage, with John Heitinga and Magaye Gueye completing the win, although the victory was overshadowed by a double leg break suffered by Bryan Oviedo.
Charis Mavrias' first Sunderland goal ended non-league Kidderminster's brave cup run 1-0, while substitute Fraizer Campbell struck five minutes after coming on to send Cardiff City through against Bolton Wanderers by the same scoreline.
