Germany had also lost 2-1 against England in November in Berlin and Loew will need to make some adjustments before his team's 2010 World Cup qualifiers in March and April.
"This is not an undeserved defeat," Loew said of the defeat in Dusseldorf's LTU Arena in front of 42,000 fans.
"Our game was slow, too dependent and not inspirational enough. We must draw our lessons and will approach our next games differently."
Norway, missing several key players including John Carew and John Arne Riise, beat a lacklustre Germany side that hardly managed to get near the box despite the return of captain Michael Ballack and fellow midfielder Torsten Frings.
Germany, finalists at Euro 2008, lacked spark and creativity despite the pair's return.
The hosts carved out only a few chances and came close once in the first half when Miroslav Klose headed on to Mario Gomez but his volley floated over the crossbar.
Germany were lucky not have a penalty awarded against them when Heiko Westerman clumsily challenged Daniel Braaten in the box in the 21st.
The Norwegians, who relied heavily on quick counter attacks, should have done better two minutes later when Thorstein Helstad completed a swift move with a powerful header from inside the box but keeper Rene Adler reacted well.
Until his substitution in the 69th Gomez was the only real threat for the Norwegians who had successfully shut out Klose with tight defending.
TOO SEDATE
Norway grabbed the deserved winner in the 63rd when a crowded Germany defence, looking shaky throughout the match, failed to clear and Morten Gamst Pedersen crossed to Grindheim at the far post who tapped it in.
"We must try to play faster against such a defensive team," Ballack said. "We were far too sedate."
Loew made several substitutions, including bringing on talented Mesut Ozil and Andreas Beck for their first appearances.
Ozil was the subject of a tug-of-war in the past weeks between Germany and Turkey because of his family roots but he opted to play for Germany.
Germany, who are on track to reach the 2010 World Cup finals in South Africa next year, face Liechtenstein in Group Four on March 28 and then travel to Wales for another qualifier on April 1.
They top the group with 10 points, four ahead of Russia who have a game in hand.
Norway are bottom of Group 9 on two points from three matches.
