The All Whites, ranked a lowly 78th in the world, snatched a last-gasp equaliser against Slovakia on their return to the sport's biggest stage having been thumped in all three games on their only other appearance at the finals 28 years ago.
Their 1-1 draw, thanks to a Winston Reid header in the third minute of added time, leaves Group F wide open after world champions Italy drew by the same score with Paraguay on Monday.
It has left a country better known for its formidable All Blacks rugby team with the chance to cling to the dream of an unlikely place in the second round.
Asked if this was New Zealand soccer's greatest moment, coach Ricki Herbert said: "It would have to be. We've never picked up a point in a World Cup. We've thrown some really good punches tonight."
Herbert was a 21-year-old defender in the 1982 side and felt the pain of heavy defeats by Scotland, the Soviet Union and Brazil.
SPECIAL NIGHT
"Tonight is a special night for me. I couldn't be any prouder. The sport I love will be leading (the back) pages," he told a news conference after the game. "We'll keep on dreaming because we're here."
New Zealand next face Italy on Sunday before taking on Paraguay on June 24 with renewed hope.
The team have resented their status as the tournament outsiders with pundits outside New Zealand, and some within, expecting them to prop up the group.
"I think the press had written us off from day one," forward Chris Killen told Reuters. "They said we had come here to make up the numbers and that's not the case".
"It's brilliant for New Zealand football, the first point we've ever picked up in the World Cup and something we can be proud of."
However, he said they would not get carried away by the result and knew they faced a hard challenge against Italy.
New Zealand fans in South Africa in back home will toast the result for some time to come and even for some players there was time to stop and enjoy the moment.
"We've been in the ashes for quite some time and we've been wanting to rise up. Tonight we did," 28-year-old striker Rory Fallon said.
Follow FFT.com on Twitter
Join FFT.com on Facebook
