Schalke, lying 10th in the Bundesliga, will face second division Duisburg in Berlin in May after reaching the final for the first time since 2005.
The defeat left last season's domestic double winners Bayern with only the Champions League to fight for following a 3-1 Bundesliga defeat by Borussia Dortmund on Saturday that effectively ended their league title hopes.
Bayern host holders Inter Milan in the second leg of their Champions League round of 16 this month.
"We knew that we would get a chance," said Schalke coach Felix Magath. "We could have added another goal from a set piece but overall I think we deserved to go through even though we did not do much in the second half."
In a rematch of last year's Cup semi-final, visitors Schalke, missing injured Klaas-Jan Huntelaar, made the most of Bayern's sloppy passing game in the first half to dominate in midfield.
They threatened for the first time with a Raul volley and the Spaniard was on target a minute later when Benedikt Hoewedes headed on a corner and the 33-year-old striker drilled his header in from close range.
Bayern upped the tempo from the start of the second half as Schalke fell back, going close through Arjen Robben but keeper Manuel Neuer, a Bayern transfer target, twice denied the Dutchman.
The 24-year-old Germany goalkeeper was in stellar form throughout Bayern's second-half charge, ignoring hundreds of Bayern fans opposed to his transfer holding up banners reading "No Neuer".
He denied Robben again before making his best save three minutes from time, stopping a Franck Ribery shot from 10 metres to offer respite to troubled Magath.
Bayern coach Louis van Gaal, who himself was already under mounting pressure following an erratic season, said his players still could not understand how they lost the game.
"This is a big disappointment and my players can still not quite understand how it happened. But that is football, a team defending all the time they get a break and score," Van Gaal said.
Duisburg became the first second division club to reach the final in seven years when they beat Energie Cottbus 2-1 on Tuesday.
