Dimitris Salpingidis put the hosts ahead in the 22nd minute with a neat lob but Israel drew level just after half-time when Greek defender Nikolaos Spyropoulos turned Roberto Coluatti's cross past his own goalkeeper Michalis Sifakis.
Greece clinched the three points when Karagounis slotted home from the penalty spot just after the hour mark to go top of the group with eight points from four games, one point ahead of Croatia who were not in action on Tuesday.
Greece were forced into a change after 16 minutes due to an injury to Sotiris Ninis, with teenager Ioannis Fetfatzidis making his second international appearance.
The change did not affect the hosts' gameplan, with Salpingidis breaking the deadlock in the 22nd minute, outpacing the Israel defence to control Pantelis Kafes' threaded pass before lofting the ball over Dudu Aouate and into the net.
Karagounis then tested Aouate's reflexes with a stinging shot from distance as the hosts sought to turn the screw, while Israel's best chance to level came five minutes before half-time but Lior Refaelov blazed his free kick over.
Fetfatzidis proved a constant threat in the second half with his superb dribbling and direct running but he could not prevent Israel drawing level when Spyropoulos accidentally turned Colautti's low cross under pressure from Etey Schechter.
The goal prompted an immediately response from Greece, Salpingidis surging into the penalty area before being upended by Aouate, leaving Swedish referee Martin Hansson with no option but to point to the spot.
Karagounis stepped up and stroked the ball into the bottom right corner to put the home side back in front and they were able to hold on to their lead comfortably.
