Dutchman Stevens, who will be replaced until the end of the season by youth team coach Jens Keller, was in his second stint at Schalke after taking over in September 2011.
Despite leading the side into the knockout stages of this season's Champions League, Stevens paid the price for a disappointing run of no wins and four defeats in their last six league games and is leaving at the start of the winter break.
"This was a very hard decision for everyone involved given the merits the 59-year-old has earned with the club," Schalke said in a statement.
This season, Schalke started off as Bundesliga leaders Bayern Munich's main title rivals but with a busy schedule, the squad has started to show signs of fatigue as the burden of European and domestic campaigns took its toll on the players.
"We sat down this morning and had a discussion with Huub Stevens, we analysed the situation together and came to the conclusion together that it made sense to end the collaboration immediately," said Schalke sports director Horst Heldt.
"The Bundesliga is our daily bread and there we were well behind expectations," he said.
"Obviously I was surprised," said Keller. "But obviously it is not something you turn down, being part of one of the big European leagues.
"We need to talk to the boys now, take away the fear they show in their game. I want them to see the Cup tie as a final and to end the year on a positive note," said Keller.
