Holman grabbed the winner after receiving a deft chip in the box from fellow midfielder Carl Valeri to lift more than 55,000 spectators at the Melbourne Cricket Ground into a frenzy.
The last-gasp goal was a cruel ending for the All Whites who were the far more composed side and had sparked hopes of a stunning upset after a 16th-minute goal from Chris Killen.
It was an undeserved victory for the Socceroos, according to Australia coach Pim Verbeek, who slammed midfielders Vince Grella and Tim Cahill for two brutal first half tackles that sent All White Leo Bertos off the field.
"I think in the World Cup that's probably two red cards, so it's a good lesson," Verbeek told reporters.
"I cannot accept this... the players know it ... you cannot afford to go into whatever game and make tackles like that."
Three yellow cards in 10 minutes - the second a sickening studs-up tackle from Grella into Bertos's shin - capped off arguably the worst half of football the Socceroos had played in recent memory.
Bertos had barely recovered from the first tackle when Cahill's floating leg caught him below the knee and sent him to the ground.
New Zealand coach Ricki Herbert said Australia were lucky not to have been down to nine men after two incidents.
"(He's) probably a little worse for wear from those tackles but ... in a normal game we probably would have been up against nine (men)," said Herbert.
"It's quite bad but hopefully he's okay," Herbert added of Bertos, who was taken from the ground shortly before the break.
England-based midfielder Grella was largely unrepentant.
"I think maybe I went a little bit too hard and it was probably not needed in a friendly game but it's a part of the football, and I take full responsibility for what happened."
Verbeek, clearly impatient, made a raft of substitutions after the interval and was rewarded with more dynamism up front.
Dario Vidosic capitalised on a cruel deflection off Killen to steer the ball into the corner of the net for an equaliser in the 57th minute to lift the home side.
"If we played the World Cup in three or four days, I should get worried, but we still have three weeks so we're still on track," Dutchman Verbeek said ahead of the June 11-July 11 finals in South Africa where Australia face Serbia, Ghana and Germany in Group D.
"I don't think (we deserved the win) ... 1-1 was more than enough, I think."
Follow FFT.com on Twitter
Join FFT.com on Facebook
