Giovinco slotted home the winner in the 86th minute after latching onto a cross in the area, capping a pulsating contest that saw the Azzurri edge ahead with three consecutive goals after trailing 2-0 before half-time.
"We were lucky, Japan played a lot better than we did," Italy manager Cesare Prandelli told reporters.
"But we didn't give up and tried to get the goals back and even when they equalized at 3-3 we still had the drive to keep attacking."
Shinji Okazaki nodded a lovely, glancing header past Italy keeper Gianluigi Buffon from a Yasuhito Endo free-kick to level at 3-3 in the 69th minute, and Japan tried desperately to respond to Giovinco's late score in a frenetic finish but to no avail.
Italy join Brazil in the semi-finals and might consider themselves lucky after starting poorly and benefiting from an own goal by Japan defender Atsuto Uchida.
Coach Alberto Zaccheroni worried his side might be overawed by the four-times world champions but they began confidently and had their first goal in the 21st minute when Keisuke Honda slotted home a penalty kick after Buffon was adjudged to have brought down Okazaki.
The Blue Samurai doubled their lead in the 33rd minute after the Italian defence missed a high ball and it fell to Shinji Kagawa who swiveled and hammered it home.
Italy pulled one back in the 41st minute when de Rossi rose to powerfully head home a Pirlo corner and almost had an equaliser on the stroke of half-time when Emanuele Giaccherini hit the post after Mario Balotelli had set him up with a deft header.
Italy's second goal came just five minutes after the restart when Maya Yoshida was caught in possession by Giaccherini and Uchida turned his cross ball into his own net with Balotelli waiting to pounce.
Italy took the lead in controversial circumstances just two minutes later after Makoto Hasebe was harshly punished for a handball in the box and Balotelli made no mistake from the spot.
Okazaki's header gave Japan renewed hope and they were unlucky not to grab a fourth goal when they hit the post and bar in a matter of seconds.
But sloppy defending opened the door for Giovinco to grab the clincher with four minutes left in regular time.
"This match took me back to the Italy-Germany game in the 1970 World Cup. This was end to end stuff with constant changes and it was really exciting," said Japan coach Alberto Zaccheroni.
"Usually when you lay like this you win so I feel sorry for my players. I feel also that the team that won was the team that deserved it the most.
"We need to acknowledge that Italy are better at seizing their chances, they had fewer chances than us but they were more clinical in taking them."
