"The red light has been switched on for Brazil which is not meeting deadlines," Valcke told Brazilian reporters in South Africa where this year's World Cup starts next month.
"It's incredible how late Brazil is, and I'm talking about all the stadiums. A lot of deadlines have passed and nothing has happened. Brazil is not on the right path."
"This year, there's a presidential election so almost nothing will happen. Next year, Carnival comes along. So everything's only going to start after Carnival?"
"It's necessary to make the most of the time available. We don't want a rushed World Cup, with everything done at the last minute.
Valcke said he had received a report earlier on Monday which warned work was behind schedule at a number of venues.
"Brazil has been wanting to host the World Cup for a long time but now it has to start moving. They have to do this for South America, not just for Brazil."
"There's no point in just sending letters," he added in comments widely quoted in Brazil on Monday. "Very little has been done. It's time to act."
Five-times World Cup winners Brazil, which hosted the 1950 tournament, was awarded the event unopposed in 2007.
The 2014 World Cup was earmarked for South America under FIFA's rotation system and Brazil had been considered almost certain hosts since 2003 when the other nine South American federations agreed at a meeting in Asuncion to back their bid.
During the run up to FIFA's decision in 2007, Brazil concentrated heavily on its footballing past and achievements on the field but gave little away about how it would deal with the lack of infrastructure and soaring crime rates.
Brazil will host the 32-team tournament in 12 cities which many critics feel is overambitious given the lack of transport options.
