Magath, who also secured a place in the German Cup final with a win over holders Bayern Munich this month, had a contract until 2013, with the club deciding to bring it to a premature end for reasons they would not disclose.
"There are very good reasons for this separation," Schalke boss Clemens Toennies said on the club website.
"We will not comment on them, however, as there is an ongoing legal procedure."
Magath joined in 2009 with the stated goal of winning Schalke's first German league title since 1958 within the four years of his original contract.
Schalke had considered parting ways with Magath last week, with Toennies saying it was necessary "to draw a line now as the club is on fire".
In the end, the club's surprise Champions League qualification at the expense of Valencia served only to postpone the decision.
Magath's lawyer Ralf Hoecker said his client, renowned for his gruelling training methods and strict coaching regime, had not received an explanation for his sacking.
This season Schalke have languished in mid-table with fans of the success starved, fiercely supported club complaining about the coach's far reaching powers and controversial transfers.
Magath had to endure the ire of his own fans with huge banners reading "Magath out" and "Stop the madness" being unfurled at their home stadium in Gelsenkirchen, although the mood of some supporters appeared to have shifted with the progress in cup competitions.
