USA led 2-1 thanks to goals from Aron Johannsson and Jozy Altidore sandwiched in between Nicklas Bendtner's first-half opener at NRGi Park in Arhus.
But Bendtner scored two goals during the final seven minutes to condemn USA to their fourth defeat in five matches.
It was the third time USA have lost a match from a winning position recently, after also leading in the second half of friendly losses to Colombia and Chile.
"Obviously, what we don't like are late goals because of losing focus for a second or losing your marker or heavy legs," Klinsmann said post-game.
"You never want to give a game away where you're up 2-1. Overall, it's a bit frustrating in the end, especially when you lose last minute."
Klinsmann added: "There's still stuff to improve, but overall I think they did a lot of really great work tonight.
"There was a lot of good stuff taken from this game, definitely. What we need to do is step it up in terms of managing the game all 90 minutes."
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.