Magath, who also secured a place in the German Cup final with a win over holders Bayern Munich this month, had a contract until 2013, with the club deciding to bring it to a premature end for reasons they would not disclose.
"There are very good reasons for this separation," Schalke boss Clemens Toennies said on the club website.
"We will not comment on them, however, as there is an ongoing legal procedure."
Magath joined in 2009 with the stated goal of winning Schalke's first German league title since 1958 within the four years of his original contract.
While they have performed above expectations in the Champions League, their Bundesliga form has been poor this season, with the club currently down in 10th place.
Fans have also been angered by a transfer policy that has seen 40 players signed in the last 18 months.
Schalke had considered parting ways with Magath last week, with Toennies saying it was necessary "to draw a line now as the club is on fire".
In the end, the club's surprise Champions League qualification at the expense of Valencia served only to postpone the decision.
Ralf Rangnick, a former Schalke coach, has been mentioned by local media is a favourite to return.
GRUELLING TRAINING
Magath's lawyer Ralf Hoecker said his client, renowned for his gruelling training methods and strict coaching regime, had not received an explanation for his sacking.
Magath, who scored the winning goal for Hamburg SV in the 1983 European Cup final, won back-to-back league and cup doubles during his time as Bayern Munich coach and joined Schalke after steering VfL Wolfsburg to their first Bundesliga title in 2009.
Things looked to be going his way last season when the debt-ridden club snatched second place to qualify for the Champions League.
This season Schalke have languished in mid-table with fans of the success starved, fiercely supported club complaining about the coach's far reaching powers and controversial transfers.
Magath was also the team manager and a board member and many supporters were angry with his decision to relieve a long-standing fan representative of his board duties.
Magath had to endure the ire of his own fans with huge banners reading "Magath out" and "Stop the madness" being unfurled at their home stadium in Gelsenkirchen, although the mood of some supporters appeared to have shifted with the progress in cup competitions.
