Ernesto Valverde's team, who slumped to a 1-0 home defeat by Olympique Marseille in the Champions League in their first competitive match of the season last week, fell behind to a stunning 14th-minute strike from Brazilian midfielder Marcelinho.
The visitors held their own in the face of intense pressure until the 86th minute when Marco Pantelic leveled the match with a far post header.
Rafik Djebbour (pictured) then clinched victory in the fourth minute of added-time with a powerful header from a corner.
The Greek champions were forced to sit out the first two rounds of fixtures because of postponements after the teams they were scheduled to face, Kavala and Volos, were relegated to the fourth division by the Professional Sports Committee (EEA) after failing to break ties with owners who have been implicated in match-fixing.
The same panel has not yet approved the licences of Doxa Dramas and Levadiakos, who are due to take the places of the two demoted teams, meaning that currently only 14 of the 16 teams have been confirmed to play in the Greek top flight.
The EEA's examination of the applications of the two second division clubs is expected to conclude on Tuesday.
Panathinaikos, who were held to a 1-1 draw at Atromitos on Saturday, continue to lead the standings with seven points from three games.
The Greens had to play virtually the whole match with 10 players after goalkeeper Orestis Karnezis was sent off in just the third minute for handling the ball outside the area.
Atromitos struggled to make their advantage count but Kostas Mitroglou eventually opened the scoring for the home side in the 54th minute with a powerful header.
Against the odds, Panathinaikos managed to equalise just four minutes before the final whistle when a clever cross from Quincy was slammed home by Sebastian Leto, the Argentine's third Super League goal in three games.
