Stoke had been 3-1 up on Tuesday, following the first-half dismissal of Birmingham’s Wade Elliott, but two late Peter Lovenkrands goals took the game to extra time.
Kenwyne Jones then put Stoke back in front but substitute Olly Lee got a 118th-minute equaliser, his first senior goal, to take the game to penalties.
Birmingham missed their first two spot kicks, Mitch Hancox and Callum Reilly off target, and though they converted their third and fourth, Stoke scored each of their first three, leaving Steven N'Zonzi to step up and fire home the winner, sending the visitors through to just their second League Cup quarter-final since 1978-79.
Stoke boss Mark Hughes made five changes from Saturday’s defeat at Old Trafford and two of his inclusions - Marc Muniesa and Ousama Assaidi - combined to put the Premier League outfit in front after nine minutes.
Muniesa ran onto Marko Arnautovic's perfectly weighted throughball, before back-heeling into the path of Assaidi who fired an excellent 20-yard strike straight into the top corner for his first goal for the club.
The hosts drew level 28 minutes in when Tom Adeyemi got onto a Nikola Zigic knockdown and finished well past Thomas Sorensen.
But, in first-half injury time, a clash between former Burnley midfielder Elliott and Assaidi left the Moroccan winger in a heap and his Stoke team-mates incensed.
Stoke took full advantage of the dismissal when Peter Crouch met Muniesa's whipped cross, steering his header back across Colin Doyle and into the far corner of the net.
Arnautovic looked to have put some gloss on the result with a superb strike from just inside the box after latching onto N'Zonzi's pass with 19 minutes remaining.
Hughes' men looked to have done enough but, with three of the five minutes of injury time gone, Lovenkrands latched onto Demarai Gray's pass and fired home a sensational equaliser.
