Speaking on the eve of Croatia's decisive Euro 2012 Group C match against European and world champions Spain, Bilic warned them to stay away from the stadium.
"It's very disappointing to hear that some of our supporters have behaved like this," Bilic told a news conference at the Gdansk Stadium on Sunday.
"I don't like these people, I don't want these people supporting us. We don't want them in the stadium. We are angry with these stupid fans as well. We need to stop them forever."
Bilic, who played in England for West Ham United and Everton and was part of the Croatia side that came third at the 1998 World Cup in France, denied that racism was endemic in Croatia.
"That's my personal opinion," he said. "I think that there is a problem all over Europe and the world.
"But as a Croat, as a parent as a sportsman who comes from a modern, open-minded country, I don't think there is a specific problem in Croatia with racism. We are a tolerant society."
UEFA charged Croatia's Football Federation (HNS) for alleged racist chants and symbols displayed by the national team's fans during the Group C match against Italy in Poznan on Thursday.
A leading independent anti-racism organisation which is monitoring crowds at Euro 2012 also said hundreds of Croatian fans had racially abused Italy striker Mario Balotelli.
UEFA is investigating reports that a banana was thrown on to the pitch in the same game, and is looking into racist chanting during two other matches at the tournament.
Croatia will reach the quarter-finals as group winners if they beat Spain on Monday while a draw of 2-2 or higher would also take them through to the knockout phase.
