The three-time Asian champions were seconds away from a shock defeat to a side ranked 104th in the world, 75 places below Japan in FIFA's rankings, when defender Yoshida headed home skipper Makoto Hasebe's cross unmarked at the far post in their opening Group B match at the Qatar Sports Club Stadium.
Jordan, in only their second appearance in the continent's major championship, looked set for a deserved win after taking the lead through Hasan Abdel-Fattah just before half-time.
Abdel-Fattah's shot from the edge of the box took a deflection off the foot of Yoshida, who thought he had scored after 25 minutes before his effort was ruled out for an infringement.
"I could not have imagined this result," Japan's Italian coach, Alberto Zaccheroni, told reporters.
"We had expected to win. We had prepared well. We had all the possession in the first half and forced Jordan to play defensively.
"We need to improve in front of goal. I am pleased we made chances, but we need to take them."
Jordan's Iraqi coach, Adnan Hamad, said he was sad that his side could not hold on to their lead and win.
"We had victory within our grasp - it was a sad result in the end, but not a bad result," he said.
The match began in hot, bright winter sunshine in the modest Qatar Sports Stadium dominated by Doha's burgeoning skyscraper skyline.
Japan, aiming for an unprecedented fourth title, settled first and spent most of the opening 45 minutes keeping possession almost at will in Jordan's half.
