Defender Franz Schiemer put Austria 3-2 ahead just after the hour with his second goal of the match but the visitors had Paul Scharner sent off six minutes later for a petulant foul committed right under the referee's nose.
They held on until the 87th minute when Marvin Ogunjimi fired the equaliser before Nicolas Lombaerts headed Belgium in front in the 90th.
In between, there was also time from Belgium's 17-year-old forward Romelu Lukaku to hit the inside of the post.
A dramatic final twist saw substitute Martin Harnik, who had been on the pitch for five minutes, break clear to complete the scoring three minutes into added time.
The result left the Austrians unbeaten in Group A with seven points from three games and Belgium with four from four. Germany lead the group with a maximum 12 points from their four matches.
Marko Arnautovic, considered the bad boy of Austrian football, also scored, taking his tally to three in his last two internationals following his recall after an 18-month absence.
But the lanky 21-year-old showed the other side of his game, wasting an outstanding chance to make it 4-2 by shooting instead of passing to an unmarked team mate on the counter-attack.
Jelle Vossen gave Belgium an 11th minute lead before Schiemer and Arnautovic put Austria in front before the half hour.
Austria goalkeeper Jurgen Macho then saved two Vossen shots with his legs before Axel Witsel struck the bar from the rebound.
There was no let up after halftime as Macho again saved with his feet, this time denying Witsel, only for Marouane Fellaini to head the equaliser from the resulting corner in the 47th minute.
Schiemer scrambled the ball home in the 62nd minute to put Austria 3-2 ahead and the visitors wasted a number of counter-attacks before the real drama unfolded at the end.
Thank you for reading 5 articles this month* Join now for unlimited access
Enjoy your first month for just £1 / $1 / €1
*Read 5 free articles per month without a subscription
Join now for unlimited access
Try first month for just £1 / $1 / €1
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.