Ivory Coast were livid at Luis Fabiano's second goal for Brazil after replays showed that he twice handled the ball before crashing it into the net.
Brazil, 3-1 winners, were furious after Kaka was sent off late in the game after picking up two yellow cards in quick succession, the second harshly awarded after an altercation with Kader Keita.
"I heard that Brazil complained a lot, they shouldn't complain," said Ivory Coast coach Sven-Goran Eriksson.
"It's difficult to cope with Luis Fabiano and even more difficult if he's allowed to use its hands. Of course, it's handball, not once by twice.
"They got a free goal which made it 2-0 goal and changed everything.
Ivory Coast captain Didier Drogba added: "It's a big disappointment, particularly the way the Brazilians got their second goal. It was clear it was handled.
"A 2-1 scoreline would have been fairer."
But Brazil coach Dunga blamed the referee for allowing the Africans to get away with rough play and said Kaka's red card was unfair.
Having been booked, Kaka was then given a second yellow card for after Keita ran into him and keeled over clutching his face.
"It was totally unjust, he was the one who suffered the foul," said Dunga after replays showed that Kaka had slightly raised his arm but done little more.
"We all like to see a good spectacle but the people who have to control the game have to know what's football and what isn't," he said.
"It's difficult to play attractive football when the ref lets things go like he did today.
"We ended up with more yellow cards than the other team who kept fouling us, so what can we do?"
A hard-tackling midfielder in his playing days, Dunga added: "It would have been good for me to be able to foul like that without getting penalised."
He added: "Today was a tough test for the players who showed maturity.
"They were kicked from start to finish and never retaliated. They mustn't think about kicking, complaining, they have to be focused on just playing football, on what they do best."
Kaka, who was twice sent off when he played for Sao Paulo, did not want to comment on the incident.
"I prefer to let the pictures speak for themselves," he told reporters. "You can analyse what happened, I've already had thousands of messages of support."
‘After Manchester City’s recent form, maybe they’re the underdogs against Manchester United!’ Former Red Devils defender on this weekend’s derby
‘Arteta, Alonso, Emery, me… none of us were physical players – we needed the understanding of the game. That probably helped us move into management’: Premier League boss reveals reasons for natural career progression
