Both sides advanced from Group B after Hapoel Tel Aviv's victory over Benfica left the Portuguese side out of contention.
Schalke, despite some inconsistent performances in the Bundesliga in which they sit 15th, produced their most impressive display of the season against Lyon.
"There is more harmony in the team and that is what has been lacking at the start of the season," a beaming Schalke coach Felix Magath said. "This win was a sign that we are getting stronger.
"We have become more stable, more consistent and to keep a clean sheet against such a strong team means we are finding ourselves, and we see that the season is not as bad as the table in the Bundesliga shows," he added.
With winger Jefferson Farfan combining brilliantly with front pair Raul and Klaas-Jan Huntelaar, Schalke's attack ripped into a dazed Lyon defence.
But it was their own back-line, that had let them down so often this season, that lived up to the occasion with former Germany international Christoph Metzelder and Bendikt Hoewedes shutting down the corridors for the usually aggressive French outfit.
Schalke's early pressure paid off in the 14th minute when Farfan benefited from a Raul effort in the box to fire them into the lead.
"Our start was very good. We kicked off very well and showed in the opening minutes we wanted to win and to carve out an early lead," said Magath.
They did just that as Lyon struggled to contain Huntelaar and Raul, with the Dutch striker netting Schalke's second goal in the 20th minute after a perfect pass from Peer Kluge from the left side.
Huntelaar added another in the 89th minute with the help of a deflection to book Schalke's ticket into the next round, a far cry from the early stages of the season when they lost their first four Bundesliga matches.
"This shows that we can play football and we can do it against teams like Lyon. The only way is up and that is the direction we are taking," said Schalke goalkeeper and captain Manuel Neuer.
