Japan's Yuichi Komano hit the crossbar with his team's third spot-kick, the only one missed in the first penalty shootout of the finals.
GEAR:Get your Paraguay shirt here
Paraguay, who became the fourth South American team in the last eight joining Argentina, Brazil and Uruguay, will now meet Spain or Portugal.
"God was on our side. Now we want more," said Paraguay coach Gerardo Martino, who was in floods of tears at the end of the shootout.
"Thank god we got through, Paraguay have never gone so far," said striker Oscar Cardozo, who scored Paraguay's winning penalty.
LIVE:Minute-by-minute coverage
"We are all very happy, me because I scored the penalty and got it in, thank God. It was a very difficult match because Japan too have very good players."
The South Americans had enjoyed more possession during the 0-0 draw which produced relatively few clear goal chances at either end.
A cagey opening brought few clear-cut chances but the game sparked to life in the 20th minute, Eiji Kawashima saving with his knee from Lucas Barrios, then Daisuke Matsui cracking the Paraguay crossbar with a curling 25-metres effort.
An out-of-sorts Honda had his first clear sight of goal in the 38th minute, striking the ball first time with the outside of his favoured left foot just past the post.
Paraguay looked set to open the scoring 10 minutes into the second half when Edgar Benitez broke into the left side of the penalty area but his shot was deflected wide by Yuji Nakazawa.
Japan, who had torn Denmark apart in their final group game with some delightful free-flowing football, struggled to maintain possession in the second half and their trademark quick-fire counter-attacks fizzled in the final third.
With the score tied at 0-0 after 90 minutes, Japan made a bright start to extra time, Yoshito Okubo bearing down on the Paraguay defence with a surging run from deep in his own half.
Paraguay almost unlocked the Japanese defence seven minutes into extra time when Claudio Morel released substitute Valdez in the box but goalkeeper Kawashima got out smartly to block.
Paraguay's Edgar Barreto had his head in his hands moments later when the ball dropped to him in a crowded penalty area but he could do no better than flick the ball over the bar.
‘After Manchester City’s recent form, maybe they’re the underdogs against Manchester United!’ Former Red Devils defender on this weekend’s derby
‘Arteta, Alonso, Emery, me… none of us were physical players – we needed the understanding of the game. That probably helped us move into management’: Premier League boss reveals reasons for natural career progression
