The La Liga pair joined Atletico Madrid in being humbled by lower league opposition after Quique Sanchez Flores's side had Tomas Ujfalusi sent off on Wednesday as they were thumped 3-0 at second-tier strugglers Recreativo Huelva.
Mallorca, fourth in Spain's top flight, failed to reproduce their impressive league form at Rayo's snowy Teresa Rivero stadium in Madrid and went behind after a quarter of an hour.
Juan Jose Collantes headed in from a tight angle to restore Rayo's lead two minutes before halftime and Mallorca had a lucky escape late on when defender Enrique Corrales cleared a shot off the line.
Striker Jorge Galan levelled shortly after the break before Joseba del Olmo smashed the winner in off the bar with just over 20 minutes remaining to give the second division leaders the advantage ahead of next week's second leg in Pamplona.
In Thursday's other first leg, relegation-threatened Malaga beat visiting Getafe 2-1 in an all-La Liga clash at a half-empty Rosaleda stadium.
Apono surprised Getafe keeper Jordi Codina with a shot from distance that went in off a post after nine minutes and Edinho's wickedly swerving freekick doubled the home side's lead a minute before the break.
Getafe were gifted a potentially valuable away goal with 23 minutes remaining when Malaga's Milan Stepanov attempted a clearance that looped over his own keeper Roberto Santamaria.
In Wednesday's other matches, Deportivo La Coruna snatched a 2-1 win at La Liga rivals Valencia, and Villarreal drew 1-1 at second division Celta Vigo.
