Defender Franz Schiemer put Austria 3-2 ahead just after the hour with his second goal of the match but the visitors had Paul Scharner sent off six minutes later for a petulant foul committed right under the referee's nose.
They held on until the 87th minute when Marvin Ogunjimi fired the equaliser before Nicolas Lombaerts headed Belgium in front in the 90th.
In between, there was also time from Belgium's 17-year-old forward Romelu Lukaku to hit the inside of the post.
A dramatic final twist saw substitute Martin Harnik, who had been on the pitch for five minutes, break clear to complete the scoring three minutes into added time.
The result left the Austrians unbeaten in Group A with seven points from three games and Belgium with four from four. Germany lead the group with a maximum 12 points from their four matches.
Marko Arnautovic, considered the bad boy of Austrian football, also scored, taking his tally to three in his last two internationals following his recall after an 18-month absence.
But the lanky 21-year-old showed the other side of his game, wasting an outstanding chance to make it 4-2 by shooting instead of passing to an unmarked team mate on the counter-attack.
Jelle Vossen gave Belgium an 11th minute lead before Schiemer and Arnautovic put Austria in front before the half hour.
Austria goalkeeper Jurgen Macho then saved two Vossen shots with his legs before Axel Witsel struck the bar from the rebound.
There was no let up after halftime as Macho again saved with his feet, this time denying Witsel, only for Marouane Fellaini to head the equaliser from the resulting corner in the 47th minute.
Schiemer scrambled the ball home in the 62nd minute to put Austria 3-2 ahead and the visitors wasted a number of counter-attacks before the real drama unfolded at the end.
