The hosts, playing at the compact Bloomfield Stadium, delivered a display to send the fans home with a smile and lift the pressure on embattled coach Luis Fernandez.
Israel have struggled since the Frenchman took over a year ago but he fielded a number of new, young faces against Latvia and the move paid off.
Fernandez (pictured) said Tuesday's game against Georgia would be a much tougher challenge.
"The Georgians are a tough side, they beat Croatia 1-0 today and they are capable of playing good football," the Frenchman told reporters.
"It will be very tough for us and for some players to turn out for a second time in four days will be very difficult."
WORLD CUP
The margin of victory against Latvia could have been bigger but for some wayward finishing from Israel who last qualified for a major competition when they played at the 1970 World Cup finals in Mexico.
Elyaniv Barda scrambled the ball over the line to put the home team in front after 16 minutes but Latvia equalised when Kaspars Gorkss took advantage of a defensive mix-up to score in the 62nd.
Israel though increased the tempo as Fernandez introduced two attacking substitutes, Ben Sahar and Maor Buzaglo, and it was Buzaglo who set up the 81st-minute headed winner for Kayal.
The victory gave Israel seven points from five games and kept alive their chances of qualifying. Greece lead the group with 11 points from five matches.
Fernandez said he was particularly pleased with the performance of debutant left back Taleb Tawatha who was a revelation with his determined runs up and down the touchline.
"He's the Israeli Roberto Carlos," added the Frenchman referring to the great Brazilian left back.
Latvia coach Aleksandrs Starkovs was disappointed with his team.
"It's not the result we hoped for but it properly reflects the game," he said.
