Milan Jovanovic grabbed the winner in the 38th minute with Germany in disarray moments after striker Klose was sent off for a foul that brought him a second yellow card in an ill-tempered Group D match at Nelson Mandela Bay Stadium.
Germany should have equalised in the 61st when Nemanja Vidic needlessly handled in the area but goalkeeper Vladimir Stojkovic dived to his left to save Podolski's poor spot kick.
Germany still lead Group D with three points from two games, ahead of Ghana and Serbia, also on three, and Australia on none. Ghana and Australia meet on Saturday in their second game.
"This victory is more than just a victory. This victory has a much greater impact on us psychologically," said Serbia coach Radomir Antic. "With this victory we can decide our own fate. This is a victory for our people who'll know how to celebrate."
LIVE:Minute-by-minute coverage
Until Klose's dismissal, the Germans had been in control of fiercely contested match, playing with attacking flair, fast copunter-attacks and a fluid movement off the ball that left Serbia on the back foot and staring at an early exit.
The three-times World Cup winners' spurned chance after chance, though, with Sami Khedira volleying over after a neat cross by Thomas Mueller before rattling the crossbar with a ferocious long-range drive in the second half.
DANGEROUS ATTACKS
The Germans thrilled the crowd with classy interplay between Klose, Podolski, Mueller and Bastian Schweinsteiger and they overran the Serbia defence with a torrent of dangerous attacks.
The Serbs had only a few chances but went close early on when Milos Ninkovic found himself with time and space but blasted Milos Krasic's inch-perfect cross over the bar while Nikola Zigic's header clipped the upright.
Klose's frustrations mounted after his missed chances and a disallowed goal and he was given his marching orders after a receiving a second booking for bringing down Milos Krasic.
With the Germans stunned, Serbia took full advantage and snatched the lead a minute later with a superb team effort that culminated in Jovanovic's close-range strike.
Zdravko Kuzmanovic slid the ball into the path of Krasic, who delivered a delightful cross that was headed to the unmarked Jovanovic, who chested down and converted on the turn.
Germany battled hard to equalise but Serbia smothered their attacks and grabbed the points to stay in contention for a second round place.
"We are strong, we were a bit lucky but we deserved to win," said Ser
