Andreas Ivanschitz gave Mainz an early lead and, although Daniel van Buyten headed Bayern level after half-time, further goals from Marco Caligiuri and Niko Bungert put the hosts 3-1 up.
Van Buyten then pulled another goal back with 11 minutes left, which like the Belgian defender's first effort came from a floated Toni Kroos free-kick.
Bayern, who looked unstoppable earlier in the season when they won 10 games in a row in all competitions, have now lost twice while captain Bastian Schweinsteiger has been sidelined following surgery on a broken collar bone earlier this month.
Earlier, Werder Bremen bounced back from the previous week's 5-0 drubbing at Borussia Monchengladbach by beating VfB Stuttgart 2-0.
Bayern, top at the start of the weekend, have dropped to third with 28 points from 14 games, one behind Borussia Dortmund and Monchengladbach. Bremen are fourth with 26.
Dortmund, who beat Bayern 1-0 a week ago, continued their winning run with a 2-0 victory over Schalke 04 in the "Mother of all Derbies" on Saturday and Monchengladbach won 3-0 at Cologne on Friday.
"Not many people believed we could beat Bayern so this performance has to be judged very highly," said Ivanschitz. "I think we deserved to win."
Bayern goalkeeper Manuel Neuer sought to explain where his team had gone wrong.
"We allowed ourselves to be dragged in to playing at their hectic pace," he said. "We have so much quality in our team but today we did not show it."
DEFENCE-SPLITTING PASS
Mainz, 13th with 15 points, went ahead in the 11th minute when Nicolai Mueller split the Bayern defence and Austria playmaker Ivanschitz rounded Neuer, calmly switched the ball to his left foot and placed his shot in the far corner.
Although they never looked comfortable, Bayern levelled in the 56th minute when Kroos' free-kick was deflected in, Van Buyten getting the credit even though the ball appeared to hit a defender.
Mainz regained the lead 10 minutes later when Caligiuri caught Neuer off his guard by unleashing a low 30-metre shot which found the bottom left-hand corner.
Bungert headed the third eight minutes later before Van Buyten popped up at the far post to give Bayern new hope.
Bremen dominated the earlier game and went ahead in the 57th minute when Aaron Hunt collected Clemens Fritz's pass on the edge of the area, slipped past Khalid Boulahrouz and poked the ball under goalkeeper Sven Ulreich.
Although Bremen wasted plenty of chances, Brazilian defender Naldo made the game safe 10 minutes later with an unstoppable long-range shot following a free-kick.
