The FSS said on its website that the split was by mutual consent.
"At a meeting attended by Petrovic and FSS president Tomislav Karadzic on Friday, agreement was reached in an amicable atmosphere that the best course of action was to part company by mutual consent," the FSS said.
"This means Petrovic is not Serbia's coach as of today, with both parties concurring that further cooperation was unfeasible after the national team failed to achieve its objectives."
Former Yugoslavia and Arsenal midfielder Petrovic took over from Radomir Antic last year in the midst of Serbia's troubled campaign and made a poor start with a shock 3-1 home defeat by Estonia who went on to claim a play-off spot.
Serbia then lost in Italy by a 3-0 walkover when the match in Genoa was abandoned after six minutes when Serbian fans caused a riot by throwing flares at home fans and players while also clashing with police.
The Serbs briefly rekindled their hopes of reaching next year's finals in Poland and Ukraine after two wins over Northern Ireland and a 3-1 defeat of the Faroe Islands.
However, a 1-1 home draw with Italy followed by a crunch 1-0 defeat in Slovenia on Tuesday, when they needed a win to clinch a play-off berth, ended their challenge and also prompted captain Dejan Stankovic to quit international football.
Petrovic hinted after the match he was likely to follow in Stankovic's footsteps one way or the other and on Friday he acknowledged he had to take responsibility for the failure.
"Unfortunately, we have failed to produce the results we had anticipated and as responsible individuals, we made the only possible decision," he said.
"Being in charge of the national team was a privilege and an honour, I wish them and the FA the best of luck in the future."
