PSG, unbeaten since Ancelotti took over from Antoine Kombouare in December, took the lead through a Nene penalty but Lyon struck back before the break with goals by Kim Kallstrom and Lisandro Lopez before wrapping it up with a Bafetimbi Gomis penalty in stoppage time.
They will travel to GFCO Ajaccio in the semi-finals after the third division side knocked out Montpellier 1-0.
Stade Rennes, who claimed a 3-1 win at 10-man Valenciennes earlier on Wednesday, will visit third division side Quevilly who reached the last four by beating Olympique Marseille 3-2 after extra time on Tuesday.
"It was a great first half. Then we had to sit back in the second half because Paris were really strong but we were disciplined and that's why we held on. Hats off to the team," Lyon coach Remi Garde told France 3.
PSG went ahead in the 19th minute when the hosts were awarded a penalty after Samuel Umtiti handled the ball.
Brazilian Nene converted from the spot but Lyon were quickly back into contention as Kim Kallstrom fired a 25-metre free-kick into the top corner to make it 1-1 after 25 minutes.
Lyon took the lead six minutes before half-time when Lisandro latched on to a Kallstrom cross at the far post to double the tally from close range.
France striker Gomis put the result beyond doubt from the spot after Michel Bastos had been brought down in the box by Christophe Jallet.
Montpellier fell to Yohan Bocognano's crossed shot in the 75th minute after a mediocre performance in Corsica.
"It's wonderful, we just beat a team who are in the race to the French title," Bocognano told French TV channel France 2.
Ligue 1's top striker Olivier Giroud hit the woodwork with five minutes left but that was all Montpellier had to show for themselves four days after they suffered their first defeat of the year in a 1-0 loss at Nancy.
Rennes prevailed thanks to second-half goals by John Boye and Youssouf Hadji after Valenciennes were down to 10 men following Foued Kadir's dismissal in the 42nd minute for a rough tackle on Yann Mvila.
Valenciennes striker Mamadou Samassa had cancelled out Jonathan Pitroipa's first-half opener.
