Sports Minister Aldo Rebelo told a news conference that Valcke's words on Friday had been "offensive" and "unacceptable."
"The government will no longer accept... [Valcke] as an interlocutor," Rebelo said. "The FIFA secretary's declarations... make it difficult to have an environment of cooperation and understanding."
Valcke, who was due to visit Brazil in coming days, had said "not a lot is moving" in the country's preparations. He dismissed Rebelo's reaction as 'puerile'.
"Why doesn't he deal with the issue?," he replied when informed of the comments during a news conference in England where he was attending the annual meeting of FIFA's International Football Association Board.
"I'm fine. If that's the problem because nothing has happened over the five years the result of one remark I made, then, wow.
"One comment saying things are not working well, and I for once say exactly what is happening in Brazil. If the result is they don't want to talk to me any more, I'm not the guy they want to work with - that's a bit puerile."
Brazil's Congress has angered FIFA by dragging its feet in passing laws relating to the sale of alcohol at the games, as well as which groups will receive discounted tickets.
Rebelo said in separate comments on Friday that Brazil's government still wants native Indians and low-income families to be able to afford the games, accusing FIFA of wanting only to make a profit.
Making matters more complex, the disputes have come at a time of turmoil for Brazilian football. The head of the country's football confederation, Ricardo Teixeira, faces corruption allegations and constant rumors that he may be forced out.
The stakes for Brazil are enormous. President Dilma Rousseff has made public-works projects for the Cup a personal priority, and Olympic officials are also watching Brazil's progress closely ahead of the 2016 Games in Rio de Janeiro.
"We have compromised on a number of things such as ticket price discounts for elderly people and a number of things on the wording of these guarantees but some are key," he said.
"There is no way we cannot receive a guarantee on security because action against terrorism is a responsibility of the state: We have no leverage on this so this World Cup
