Gekas opened the scoring with a fourth-minute header from a cross by Georgios Karagounis.
Maris Verpakovskis, who plays his club football in Greece with Ergotelis, levelled with a superb bicycle kick eight minutes later before putting Latvia ahead just before halftime.
"We had said it was an all or nothing game," said Greece coach Otto Rehhagel. "The game began as we had all imagined with a great goal from Gekas and we thought it would continue that way.
"Going in at half-time nobody expected us to be behind and the team was like a punch-drunk boxer. I had a good chat with the players though and we played some great attacking football in the second half."
To go through automatically to next year's finals the Greeks must beat Luxembourg in their final game on Wednesday while the Swiss need to lose at home to Israel.
Latvia coach Aleksandrs Starkovs said: "The penalty changed the match. It was a bad decision as far as I am concerned, a real punch in the stomach for our team.
"It was a very important match for Latvia, a match which could have changed the table dramatically. We were very good in the first half ... but unfortunately it seems we can't keep this level up for a full 90 minutes."
