Serbia would have gone through to the knock-out stages of the tournament if they had grabbed a late equaliser. In the dying minutes of the game, Cahill appeared to handle the ball in the box, and the Serbians screamed for a penalty.
But Uruguayan referee Jorge Larrionda waved them away, and was surrounded by angry Serbian players after he blew the final whistle.
Serbian coach Radomir Antic was also upset his team had two goals disallowed, one of which he claimed was "definitely not offside."
"We did not get fair treatment, we deserved more than we got and we have to go home now," Antic said.
"The ref had a black day when it came to Serbia. The referee and the fourth official did not want to see a foul that was quite obvious. We are angry when it came to this decision."
With Germany beating Ghana 1-0 in the evening's other match, Australia finished level on four points with the Africans. But their goal difference was worse than Ghana's and that condemned them to a third place finish.
Serbia finished bottom of the group with three points.
Despite needing to score goals, the Socceroos were strangely subdued in the first half and the Serbians were totally in control.
Milos Krasic squandered their first chance, rounding Mark Schwarzer in the Australian goal after 12 minutes before skewing his shot wide and high.
Schwarzer produced a fine reflex save to block a close range effort from Branislav Ivanovic midway through the half, and 10 minutes later the giant figure of Nikola Zigic should have scored when left unmarked inside the box, but glanced his header well wide of goal.
The pacy Krasic, who was a menace on the Serbian right flank all evening, slipped though the Aussie defence after 38 minutes and squeezed the ball under Schwarzer into the net, but his effort was correctly ruled out for offside.
Australians heads were down as they trooped into the dressing room at half time, but they came out fighting after the break.
Mark Bresciano tested Serbian goalkeeper Vladimir Stojkovic with a long-range shot on the hour and in the 69th minute Cahill, back in the side after missing the last match through suspension, put his side ahead.
Mark Wilkshire swung in a cross from the Australian right and Cahill met it with a firm header that gave Stojkovic no chance.
Four minutes later the Australians doubled their lead - second half substitute Holman blasting home a shot from outside the box - and for a moment it seemed like Pim Verbeek's side might pull off one of the great escape acts of the World Cup.
With thousands of yellow-shirted fans roaring them on from the stands, the Australians poured forward in search of more goals.
But inevitably they left themselves exposed and in the 84th minute Serbia pulled one back, Marko Pantelic taking advantage of a rare Schwarzer fumble to stab the ball home from close range.
That changed the emphasis of the tie and suddenly it was the Serbs who had a chance of qualif
