The Austrian federation (OEFB) said that the search for a new coach would begin immediately and Constantini's successor could make his debut in a friendly in November.
"The contract which expires on December 31 this year will not be renewed," said the OEFB in a statement on its website following a meeting between Constantini, 56, and federation president Leo Windtner.
"According to the timetable, the new coach should be established by the start of November with his debut in a friendly on November 15."
Constantini, who replaced Karel Brueckner two years ago, has managed seven wins and three draws in 23 matches during which he has barred some of Austria's top players including midfielder Andreas Ivanschitz.
Tuesday's goalless draw at home to Turkey ended their chances of playing at Euro 2012 and maintained their record of having never reached the tournament through the qualifying competition.
On Friday, they were thrashed 6-2 in Germany, their sixth defeat in a row by their neighbours.
Austria's only appearance at the European championship was in 2008 when they qualified automatically as co-hosts with Switzerland. However, they were knocked out in the first round.
Austria's fortunes have waned over the last few years and they have not qualified for the World Cup since 1998.
Constantini will remain in charge for the two remaining games, away to Azerbaijan and Kazakhstan, in Euro qualifying Group A, although the OEFB said that if the new coach was found quickly, he could also take over for those games.
