Last year's FIFA Confederations Cup was blighted by demonstrations over the massive investment that has gone into the country's staging of the competition.
Further protests are still occurring in the South American nation, which will host the World Cup from June 12 to July 13, arguing in favour of increased social spending.
Brazil national coach Scolari has no issue with the protests but is concerned over the timing of the demonstrations.
"I think protests can happen," Scolari told Brazilian TV show Fantastico. "If they are normal, without smashing things up, then that is democracy.
"But I don't know if it's the right time.
"They are Brazil players and they have one mission. They can think, they can express themselves, they can say 'I want a better Brazil, too', but I don't want it to be something that causes problems in our environment."
When asked if the unrest could have an impact on his players, Scolari replied: "It could, big time."
Tournament authorities are thought to be concerned that demonstrators may attempt to disrupt some of the 64 World Cup games being held after protestors marched towards stadia during the Confederations Cup.
Brazil open the competition against Croatia on June 12 in Sao Paulo before taking on Mexico and Cameroon in their other Group A encounters in Fortaleza and Brasilia respectively.
