The Italian was already under pressure following Friday's 2-1 home loss to Sweden. Any chance of the Irish reaching the play-offs was extinguished in Vienna.
They are now six points behind second-placed Sweden with two games to go and face a daunting trip to leaders Germany next month.
FAI Chief Executive John Delaney said: "We thank Giovanni Trapattoni, Marco Tardelli and Fausto Rossi for the last five-and-a-half years during which we qualified for our first major tournament in 10 years and were close to qualification for 2010 World Cup in South Africa after the play-off in France.
"This particular World Cup campaign has been disappointing but Giovanni leaves us with a group of good young players which should form the basis of the squad that the new manager will use for the European Championship in France 2016 when 24 teams qualify."
Trapattoni took over the role in 2008, winning 26 of his 64 games in charge, leading the Boys in Green to the 2012 European Championship.
He said: "I want to thank everyone in Ireland who has given me their support during our time here, which has always meant a lot to us.
"We leave this country with emotion because we understand that Irish supporters have a well-deserved international reputation and they have our utmost respect."
Former Leicester City, Celtic, Aston Villa and Sunderland manager Martin O'Neill is the early favourite to succeed Trapattoni.
