"The 52-year-old signed a contract until 2014," the Bundesliga club said in a statement, adding that he would be on the bench for the first time against St Pauli on April 1.
Interim coach Seppo Eichkorn will take charge for Sunday's game against Bayer Leverkusen.
"Rangnick will be officially presented on Monday," Schalke said.
Rangnick's appointment ended weeks of uncertainty after the club had hinted they wanted to sack Magath following a mediocre run but the team's qualification for the German Cup final and the Champions League last eight delayed that decision.
Rangnick coached Schalke from September 2004 to December 2005, steering them to the German Cup final and the Bundesliga runners-up spot.
They also finished second last season, under Magath.
TWO PROMOTIONS
Rangnick's most recent job was with Hoffenheim where he took over in 2006 with the team in the regional division. He led them to two promotions and into the Bundesliga before leaving in January.
Rangnick's coaching style could not be more different than Magath's. The new coach is a strong advocate of attacking football while his predecessor put a lot of emphasis on tight defence.
Rangnick's Hoffenheim took the Bundesliga by storm in their debut season, topping the standings at the halfway mark, before a string injuries and inexperience took their toll in the latter stages.
He chose to sign mostly talented, young players around the age of 23 or less to give them room for development, playing what became Hoffenheim's trademark attacking football for the better part of two season.
Differences of opinion with Hoffenheim's billionaire backer Dietmar Hopp over the sale of Luiz Gustavo to Bayern Munich and a string of uninspiring performances led to his departure halfway through the current campaign.
Schalke lie 10th in the standings.
