The Uruguayan completed his brace in the 117th minute to give Laurent Blanc's men the hard-earned victory in the last-16 clash at the Parc des Princes on Wednesday.
Cavani had opened the scoring in the 25th minute and PSG looked headed for victory before Mevlut Erdinc levelled in the second half for the seventh-placed Ligue 1 side and defending champions.
PSG were knocked out by Saint-Etienne in the quarter-finals last season on penalties before Christophe Galtier's men went all the way.
It was Cavani who gave PSG the lead with a fine goal in the first half.
Marco Verratti's cross from the left found the 26-year-old free at the back post and Cavani took a neat touch inside to round goalkeeper Stephane Ruffier before placing into an open net.
Lucas Moura almost doubled PSG's lead soon after but was denied by Ruffier before Verratti squandered a decent chance late in the first half.
And they were made to pay for their wastefulness with just 12 minutes remaining when Erdinc struck for the visitors.
A scrappy passage of play in the middle third eventually led to Fabien Lemoine receiving a pass on the right and his low cross was tucked in at the near post by Erdinc.
Just like last season, the encounter looked to be headed for a penalty shootout before Cavani intervened.
Substitute Jeremy Menez found space down the right and his pass across the face of goal found Cavani – again left unmarked at the back post – to tap in an easy winner.
Elsewhere, Lyon also battled through with a 3-2 win over fellow Ligue 1 outfit Reims.
Bafetimbi Gomis, Alexandre Lacazette and Yoann Gourcuff were on the scoresheet as Lyon never trailed after going ahead on 58 minutes.
Also though to the quarter-finals are Nantes after their 1-0 win over Auxerre, Nice having brushed past Sochaux 3-0 and Marseille, who overcame Toulouse 2-1.
Daniel Wass' brace saw Evian TG past Bastia 2-1 and Bordeaux were 2-1 victors at Rennes.
In the battle between two Ligue 2 sides, Troyes needed a 90th-minute winner from Benjamin Nivet to overcome Tours 3-2.
