The three-time Asian champions were seconds away from a shock defeat to a side ranked 104th in the world, 75 places below Japan in FIFA's rankings, when defender Yoshida headed home skipper Makoto Hasebe's cross unmarked at the far post in their opening Group B match at the Qatar Sports Club Stadium.
Jordan, in only their second appearance in the continent's major championship, looked set for a deserved win after taking the lead through Hasan Abdel-Fattah just before half-time.
Abdel-Fattah's shot from the edge of the box took a deflection off the foot of Yoshida, who thought he had scored after 25 minutes before his effort was ruled out for an infringement.
"I could not have imagined this result," Japan's Italian coach, Alberto Zaccheroni, told reporters.
"We had expected to win. We had prepared well. We had all the possession in the first half and forced Jordan to play defensively.
"We need to improve in front of goal. I am pleased we made chances, but we need to take them."
Jordan's Iraqi coach, Adnan Hamad, said he was sad that his side could not hold on to their lead and win.
"We had victory within our grasp - it was a sad result in the end, but not a bad result," he said.
The match began in hot, bright winter sunshine in the modest Qatar Sports Stadium dominated by Doha's burgeoning skyscraper skyline.
INCESSANT NOISE
The 6,000 fans, which half-filled its 12,000 capacity, were mainly Jordanian supporters, and they maintained incessant noise as their team refused to be overawed by the more experienced Japan players.
Japan, aiming for an unprecedented fourth title, settled first and spent most of the opening 45 minutes keeping possession almost at will in Jordan's half.
Daisuke Matsui and Hasebe were at the heart of most of their forward flowing moves but, to their credit, Jordan, with midfielder Baha Abdelrahman working feverishly, also crafted some good chances.
At the back they defended well, with goalkeeper Amer Shafi producing some fine saves to earn the man-of-the-match award.
He did well to thwart Shinji Kagawa after 41 minutes, when the Germany-based Borussia Dortmund striker fired straight at him after another fine pass from Matsui. It proved a costly miss for Japan.
Although on the back foot for most of the half, Jordan looked dangerous twice before they took the lead. Hasan fired straight at keeper Eiji Kawashima after 29 minutes and Basem Fathi headed narrowly wide from a corner seven minutes before the break.
Abdel-Fattah was a constant threat to Japan, who will need to improve against Syria and Saudi Arabia if they are to make the last eight.
On this performance, Jordan could prove a serious threat for both of those teams in their next two games.
