The German was under pressure heading into the friendly in Paris after his side's dreadful 6-0 thrashing at the hands of Brazil in early September.
Football Federation Australia had indicated the Socceroos needed to put in a better display against France for Osieck to avoid the sack.
But, after being outplayed and outclassed, Australia were 4-0 down inside 29 minutes in Paris in another listless performance.
The FFA wasted little time in terminating Osieck's contract, with reports suggesting the move was made barely two hours after the loss.
Osieck ignored the speculation surrounding his future heading into the clash but was unable to guarantee he still had the support of his players post-match.
He said he understood he would be under pressure after his side conceded 12 goals in two matches.
It leaves the Socceroos looking for a coach exactly eight months out from the 2014 World Cup, which they have already qualified for.
Former Australia coach Guus Hiddink appears the leading candidate for the role after helping the Socceroos to the round-of-16 in Germany in 2006.
Frenchman Gerard Houllier and Spaniard Marcelo Bielsa are among others who have been linked with the position.
If the FFA opt to turn to a local name, Melbourne Victory coach Ange Postecoglou and Central Coast boss Graham Arnold are the frontrunners.
Osieck was named Socceroos coach in August 2010, leading an ageing side to the 2011 Asian Cup final, which they lost 1-0 to Japan after extra-time.
The former Urawa Reds and Canada coach won 23 of the 44 matches he led Australia in, including seeing the Socceroos to the World Cup after an indifferent qualifying campaign.
‘After Manchester City’s recent form, maybe they’re the underdogs against Manchester United!’ Former Red Devils defender on this weekend’s derby
‘Arteta, Alonso, Emery, me… none of us were physical players – we needed the understanding of the game. That probably helped us move into management’: Premier League boss reveals reasons for natural career progression
