Ajax stayed above Manchester City, who finished bottom of the group after a 1-0 defeat at Borussia Dortmund.
Jose Callejon bagged a brace for a Real side already assured of second place and Cristiano Ronaldo and Kaka weighed in with one apiece as a weakened Madrid team swept aside the visitors.
"We finished third with a young team and we should be proud," Ajax coach Frank de Boer told a news conference.
"We now have the chance to reach the Europa League final at our own stadium. It won't be easy, there are some very good teams in the competition, but it would be a great prize."
"I am disappointed we weren't at our best," he said. "Real Madrid are a great team but we had eight players who played below their level. Playing that way we will never achieve anything.
"These players have to learn to reach their best levels on occasions like these, only then can we hope to spring a surprise."
Modric then laid on Real's second in the 28th, spinning away on the halfway line and playing a long ball for Callejon who scampered into the area to score.
Kaka, wearing the captain's armband, brought the Bernabeu to its feet with a run and shot which curled into the top corner just after the re-start, earning applause from Mourinho on the touchline.
Ajax improved in the second half, however, and a weak punched clearance from reserve Real keeper Antonio Adan allowed Boerrigter to grab the visitors' a consolation goal.
Mourinho even had time to give 17-year-old youth team player Jose Rodriguez his debut, making him the club's youngest debutant in the Champions League.
"We have to respect everyone," Mourinho said, looking ahead to the last 16.
"But when you finish top of a group, you reckon you deserve a slightly weaker opponent. We will be facing a group winner and it will be tough, but this is something for
