Damjanovic converted the only goal of the game in Podgorica after the hosts were awarded a spot kick for a foul by David Addy on Filip Kasalica after just 15 seconds.
Ghana settled into the game well after that early blow, but they wasted a host of chances to draw level, with Christian Atsu guilty of missing three alone in the first half.
The 2010 World Cup quarter-finalists continued to push for an equaliser in the second half, but even the introduction of Kevin-Prince Boateng and Asamoah Gyan could not get them back into the game.
With the World Cup only 99 days away, the African nation made the worst possible start as Addy gifted the hosts a penalty in the opening seconds with a rash challenge on Kasalica.
Damjanovic made no mistake from the spot as he calmly sent Adam Kwarasey the wrong way.
After their poor start, Ghana slowly took control of the game and as they dominated possession the chances started to come - both Atsu and Michael Essien going close, although neither were able to beat Mladen Bozovic.
With the half coming to a close, Montenegro had two chances to double their lead but, after they had a penalty shout for handball waved away, Damjanovic wasted a one-on-one as Kwarasey turned his shot around the post.
Abdul Waris wasted Ghana's first sight of goal in the second half, the forward heading a cross from Albert Adomah wide without testing the goalkeeper.
Wakaso Mubarak also went close for the visitors after a neat link-up with Atsu, while Montenegro's threat came from mostly from set-pieces and counter-attacks with Kwarasey was rarely tested again after the break as they hung on for victory.
Boateng had one final chance in the closing stages as Montenegro struggled to clear a dangerous Ghana cross, but the Schalke midfielder saw his low effort well saved by Bozovic.
