Liverpool's disappointing start to the season looked set to take another turn for the worse when Marvin Emnes volleyed Swansea into a second-half lead in Tuesday's fourth-round clash.
The home side produced a storming late fightback to reach the quarter-finals, though, as much-maligned striker Balotelli came off the bench to score his first goal in nine games with an equaliser four minutes from time.
And Lovren, who like Balotelli has come in for criticism since his big-money move to Merseyside, scored his first goal for the club in the 95th minute in a stunning finale to send Liverpool through soon after Federico Fernandez was harshly shown a straight red card for a challenge on Philippe Coutinho.
It was a welcome tonic for former Swansea manager Brendan Rodgers, whose Liverpool side had failed to score in their previous two matches and had once again been toothless until Balotelli and Lovren stunned the Welsh side.
Balotelli dropped to the bench as Rodgers made nine changes to the Liverpool side that was held by Hull City on Saturday, as Rickie Lambert and Fabio Borini started up front while captain Steven Gerrard and forward Raheem Sterling were rested.
In-form striker Wilfried Bony was named among the Swansea substitutes, with Bafetimbi Gomis starting as one of five changes to the team that beat Leicester City 2-0 at the weekend.
The hosts took the upper hand after a frantic start and Lazar Markovic spurned a great opportunity to open his Liverpool account when he sliced a left-footed shot well wide 27 minutes in after a slick passing move.
Markovic then cut in from the left and picked out Borini, who guided a measured shot narrowly wide of the far post from the edge of the penalty area as Liverpool continued to dictate the play, but lacked a cutting edge.
Former Swansea loanee Borini came close again moments later, driving forward and unleashing a right-footed strike that Gerhard Tremmel got down to turn away for a corner.
Ex-Liverpool midfielder Jonjo Shelvey was given a warm reception by the home fans, but almost silenced them when his free-kick just before half-time forced Brad Jones to dive to his left to make palm the ball away.
Swansea were struggling to contain Coutinho and the Brazil midfielder ghosted into the penalty area early in the second half, but shot straight at Tremmel.
Gomis rose to meet a Shelvey corner but his header was easily claimed by Jones as the opening goal remained elusive.
Swansea were in front 65 minutes in, though, as Emnes produced a moment of class to volley into the far corner with his left foot from inside the area after the ball had looped into the air.
Liverpool were stunned, but came close to an equaliser when Glen Johnson's venomous strike produced a reflex save from Tremmel and Neil Taylor bundled the rebound away.
Coutinho had another go from long range with just over 15 minutes remaining, but Tremmel was once again able to make a routine save.
Balotelli replaced Lambert with 11 minutes remaining and rescued his side four minutes from time when he got on the end of an inviting Borini cross to find the back of the net from six yards out.
There was more drama when Fernandez was harshly shown a straight red card for a challenge on Coutinho and then Lovren headed home the winner four minutes into time added on when Tremmel misjudged a free-kick from the Brazilian.
