As a 21-year-old defender in New Zealand's only previous World Cup appearance, he burst into tears on the bus en route to their opener when he was left out of the side to face Scotland.
They lost that match 5-2 and were also thrashed by the Soviet Union and Brazil but Herbert will never forget sitting on the bench for his country's first ever World Cup match.
It was the biggest disappointment of his playing career and a spur to him and his 'All Whites' players to belie a world ranking of 78 by upsetting 38th-ranked Slovakia.
"I understand why we are the underdogs but we don't fear anyone," he said. "Anything can happen in a cup competition. If Senegal can beat France (in 2002), we can beat Italy!"
Herbert's confidence may be one of New Zealand's best weapons as he tries to inspire a team, well beaten three times in South Africa during the 2009 Confederations Cup, to rise to their biggest soccer challenge in three decades.
Their mixed form in warm-up games - beating Serbia and then losing to Chile - has given little indication of what to expect against an equally unsung Slovakia, making their first appearance in the finals as an independent nation.
CUTTING EDGE
The Slovaks proved their potential when they beat Czech Republic 2-1 away in the qualifiers and in Stanislav Sestak, who scored in both games against their local rivals, have a striker who gives a cutting edge to their 3-4-3 or 5-2-3 system.
Slovakia coach Vladimir Weiss shares Herbert's rare distinction of playing and coaching at the World Cup, having appeared in midfield for the Czechoslovakia team that reached the quarter-finals in Italy in 1990.
He is also, like Herbert, hoping his team can be one of the surprise packages of the tournament. Weiss and his men recognise Tuesday's clash will be key to their confidence and progress in a group that also includes world champions Italy and Paraguay.
Keeper Jan Mucha, outstanding in the qualifiers, said: "We are a new team, we want to surprise everyone and do our best."
Midfielder and captain Marek Hamsik added: "The most important match is against New Zealand, the first. They are strong - we saw that in their warm-up against Serbia."
New Zealand captain Ryan Nelson missed that game with an ear infection but returns against Slovakia, while vice-captain and combative midfielder Tim Brown is a doubt, having joined the squad last Thursday following surgery on a fractured shoulder.
Slovakia's experienced defender Martin Skrtel and nippy midfielder Miroslav Stoch, 20, will be fit for Weiss's usual 4-4-2 lineup after ankle and knee injuries respectively.
Follow FFT.com on Twitter
Join FFT.com on Facebook
