Rayo play in Vallecas in the south east of the Spanish capital, and Mourinho told a news conference he was disappointed only around 300 fans made the short trip on a warm and sunny Madrid afternoon.
"I find it a shame that we are playing in Madrid and there are only 300 of our supporters," the Portuguese said after Real stretched their lead over second-placed Barcelona to 13 points ahead of the champions' game at Atletico Madrid later on Sunday.
"We are the leaders and at Getafe [also in Madrid] we had 30 fans and now 300 at Vallecas," he added.
"Our objectives are for both the team and the supporters and they [the fans] need to give a bit more. Getafe and Vallecas are very close."
Mourinho also had sharp words for the fourth official at Sunday's game, calling him "the most arrogant I have encountered in my career" after his assistant Rui Faria was sent from the bench for protesting during the second half.
"His [Faria's] behaviour is the same as that of all of us on the bench," Mourinho said.
"It's normal behaviour in which he protests with politeness and respect. They sent him off and now the referee can write whatever he wants in the match report.
"There are ways of talking to people, and speaking in a pleasant tone and in a calm and polite manner is easier than being arrogant.
"But it doesn't matter. There are others who have a more positive relationship. Maybe that lad [the fourth official] has a dominating personality."
‘Maybe I’ll be signed by Wrexham, who knows? English football would suit me – if an offer came, I wouldn’t think twice, I’d go immediately’ Euro 2024 cult hero reveals his UK dream
‘Managing Leeds? It was an option that appeared, but it wasn’t the right timing. I decided it wasn’t a good idea to leave the club I was at mid-season’: Premier League boss admits to turning down opportunity to replace Jesse Marsch in 2023
