Gekas opened the scoring with a fourth-minute header from a cross by Georgios Karagounis.
Maris Verpakovskis, who plays his club football in Greece with Ergotelis, levelled with a superb bicycle kick eight minutes later before putting Latvia ahead just before halftime.
The hosts equalised with a 47th-minute penalty from Gekas after Georgios Samaras was fouled by Oskars Klava.
The Bayer Leverkusen player then sparked frenzied celebrations by heading in a Vasilis Torosidis cross 10 minutes later before Samaras converted another smart overhead kick.
Gekas sealed a memorable display by prodding home his fourth goal in the first minute of added time to push second-placed Greece three points ahead of Latvia in Group Two with one match to go.
ALL OR NOTHING
"We had said it was an all or nothing game," said Greece coach Otto Rehhagel. "The game began as we had all imagined with a great goal from Gekas and we thought it would continue that way.
"Going in at half-time nobody expected us to be behind and the team was like a punch-drunk boxer. I had a good chat with the players though and we played some great attacking football in the second half."
Former European champions Greece also have an outside chance of overtaking group leaders Switzerland, who won 3-0 in Luxembourg on Saturday.
To go through automatically to next year's finals the Greeks must beat Luxembourg in their final game on Wednesday while the Swiss need to lose at home to Israel.
The eight best runners-up across the nine European groups go through to the two-legged playoffs.
Latvia coach Aleksandrs Starkovs said: "The penalty changed the match. It was a bad decision as far as I am concerned, a real punch in the stomach for our team.
"It was a very important match for Latvia, a match which could have changed the table dramatically. We were very good in the first half ... but unfortunately it seems we can't keep this level up for a full 90 minutes."
