The 57-year-old Stevens, who led Schalke to the 1997 UEFA Cup victory and two German Cups in 2001 and 2002 as well as a second place in the Bundesliga in 2001, has signed a contract through June 2013.
He will train the team for the first time later on Tuesday.
Stevens, who holds the Schalke record for the longest coaching stint at five years and nine months, has also coached Cologne, Hamburg SV and Hertha Berlin in the Bundesliga as well as Dutch side PSV Eindhoven and Austria's RB Salzburg.
Rangnick resigned suddenly last week after only six months in charge, saying he was suffering from exhaustion and needed a break.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.