Having already astounded football fans and pundits - first securing their first ever World Cup point and then holding world champions Italy to a shock draw - the side, with no big reputations, are one game away from knockout soccer.
Coach Herbert on Monday happily smiled for an ever-increasing number of television cameras and international reporters, plotting to further shake up the World Cup.
"I can't believe what we've done, so anything is possible," he told Reuters a day after holding Italy and ahead of Thursday's game with Paraguay in which a win, or even a draw, could see them through Group F into the last 16.
That could well take them further than the multi-millionaire players of Europe's big guns, England, Spain and Italy.
New Zealand, one of the lowest ranked teams in the tournament at 78th, sit on two points, level with Italy and one ahead of Slovakia. Thursday's opponents have four points.
A win for the All Whites will see them in the next round, while a draw with goals may also be enough, depending on the result of the Italy-Slovakia game.
VARIOUS ROLES
Herbert, who was a 21-year-old defender in his country's only other World Cup final appearance in 1982, believes this team is special.
"I had 10 years with the national team as a player, and I've been with the national team in various roles for the past 10 years and I haven't seen a team like this," he said at the side's plush golf estate base near Johannesburg.
"These players are incredibly on the same page and they're a very, very proud group, just so ambitious and want to make a difference."
Herbert said the unbeaten run at this World Cup had, for him, banished the ghosts of Spain 1982, when New Zealand were thrashed in all three group matches.
"Well and truly! I was part of that and very proud to be part of that ... they were fantastic days and it was a great team but, you know, we've moved on," he said.
"We've created history by coming back again and we've created more history by getting points and last night's result, it's just an incredible position for us to be in."
New Zealand's team has no household names, with captain Ryan Nelsen of the English Premier's Blackburn Rovers the only player from a top league. The squad even boasts an investment banker in Andy Barron.
Many still play for, or were honed by, Herbert at Wellington Phoenix in New Zealand, where rugby is the overwhelming sport of choice.
Goalkeeper Mark Paston said self belief had led them this far, and the players were enjoying every moment.
"The guys are buzzing," he told Reuters.
"I think most people outside the team had written us off (thinking we would) get battered in all three games. I've personally just ignored that, it's what we're about and not what the others think."
Follow FFT.com on Twitter
Join FFT.com
