City could have theoretically gone through from Group B with a shootout victory at the TCF Bank Stadium in Minnesota, while Olympiacos had to win in 90 minutes to keep their hopes of progression alive.
A 2-2 draw in regulation time meant the match headed for spot-kicks and Anastasios Papazoglou kept his cool from 12 yards after Micah Richards blazed over the crossbar.
Stevan Jovetic had earlier continued his purple patch with his fifth tournament goal, while Aleksandar Kolarov scored from the spot after half-time, only for Dimitrios Diamantakos to equalise on both occasions.
Olympiacos almost caught City out early on when Alejandro Dominguez collected the ball in the area and steered past Joe Hart, only for his celebrations to be cut short by the linesman's offside flag.
City responded well as Samir Nasir and Yaya Toure both curled efforts wide of the right-hand post, before Kolarov skipped effortlessly past Omar Elabdellaoui but saw his shot palmed away by Roberto.
The pressure finally told in the 35th minute with a fine solo goal from Jovetic, who beat one defender and showed neat trickery before beating Roberto at his near post.
However, the lead proved short-lived as Olympiacos equalised from the next attack. Pajtim Kasami dug out a cross from the right for Diamantakos to control and force home.
The half ended on a contentious note when Jovetic was bundled over by Avraam Papadopoulos, but the referee waved away the protests.
City piled on the pressure at the start of the second half and restored their lead when Kolarov blasted a 52nd-minute spot-kick high into the roof of the net after David Silva had been clumsily felled by Kostas Manolas.
But yet again Olympiacos came back, Diamantakos grabbing his second after beating the onrushing Hart to the ball and sliding into an empty net.
Roberto made a fine one-handed stop from Silva in the closing stages as the match headed to penalties.
Two misses each meant sudden death was required and, after John Guidetti and Andreas Bouchalakis both scored, Richards put his effort well over, allowing Papazoglou to score the winner and send Liverpool through to the final.
