Both sides named strong sides ahead of their final FIFA World Cup qualifiers, with two goals in four first-half minutes from Jose Nunez and Wilson Pittoni putting Paraguay ahead before
Ilkay Gundogan and Thomas Mueller drew Germany level.
Miguel Samudio put the visitors back in front on the stroke of half-time before a second-half Lars Bender strike ensured the hosts avoided consecutive friendly defeats.
Germany manager Joachim Loew was without injured trio Mario Goetze, Julian Draxler and Bastian Schweinsteiger, but Ilkay Gundogan and Marco Reus were among the returning contingent, despite fitness doubts before the game.
Paraguay's record goalscorer Roque Santa Cruz was recalled to Victor Genes' starting line-up, while Justo Villar made his 100th appearance for Paraguay in goal.
The strength of Germany's line-up, going into the final round of World Cup qualifiers which start in September, showed how serious Loew was taking the friendly and he will have been pleased with his side's start to the game.
A Reus free-kick forced an early save from Villar before Gundogan and Klose both had efforts from inside the area.
However, Nunez gave his side the lead against the run of play after nine minutes when he dispatched a long ball past Manuel Neuer.
Klose should have equalised for Germany shortly after when he was played through by Gundogan, but, while his dink beat Villar, it also went over the crossbar.
Pittoni then doubled Paraguay's lead after 13 minutes, following poor German defending.
The Olimpia midfielder struck a half-volley after a bad clearance which deflected off Sami Khedira and past Neuer to put Paraguay in control.
The goals kept coming, although it was the hosts finally making headway. Gundogan clawed one back after 18 minutes, hitting a deft curling strike which evaded Villar, who could only get a hand to the ball.
The Borussia Dortmund midfielder showed his value at the other end, clearing a corner off the line before being forced off with a back problem to be replaced by Bender.
Germany restored parity 31 minutes in as Mueller stayed onside and controlled a lofted ball under pressure from the onrushing Villar before calmly rolling the ball past the goalkeeper.
However, Paraguay ensured they went in ahead at the break, when Santa Cruz squared for Samudio to score from close range on the stroke of half-time.
Mario Gomez replaced Klose early in the second half and he had a quick impact, forcing a good save from Villar after 57 minutes.
Fellow substitute Jerome Boateng then rifled a shot over the bar after 71 minutes before Mueller fired across goal shortly after.
However, Germany's search for an equaliser ended three minutes later when Bender latched on to a loose clearance and fired home to make it 3-3.
Andre Schurrle had a late chance but could only blaze over from close range after waves of Germany pressure.
Next up for both sides are final batches of World Cup qualifiers, with UEFA Group C leaders Germany hosting Austria in September while Paraguay, who sit bottom of CONMEBOL qualifying, host Bolivia in the same month.
