In the big derby, Olympique Lyon went third after a 2-0 win at home to Saint-Etienne thanks to Jimmy Briand's 81st-minute header and Yoann Gourcuff's injury-time strike.
The victory came despite defender Mouhamadou Dabo being sent off on 52 minutes for lunging at Banel Nicolita six minutes after coming on.
Otherwise there was little of the tension which marred Lyon's League Cup win at Saint-Etienne in midweek when the game was stopped as fans threw flares on to the pitch.
Second-placed Montpellier's 2-0 home win over struggling Nancy meant PSG stayed three points clear at the top after 12 games but the capital club were made to work for their victory although they did hit the woodwork three times.
Thomas Heurtaux's powerful header from a corner when unmarked gave Caen a shock 12th-minute lead at a rainy Parc des Princes and their mega-rich opponents needed Nene's 20th-minute penalty to equalise.
The spot-kick was given after Benjamin Nivet was dismissed for a careless foul in the box but 10-man Caen battled well until Jeremy Menez poked home on 56 minutes when the visitors stopped playing, wrongly believing the ball had gone out.
Nene added another penalty 14 minutes from time. Jerry Vandam pulled a goal back for Caen before Javier Pastore struck late as PSG recovered from their midweek League Cup defeat at Dijon.
Marseille, whose poor early season form has put pressure on boss Didier Deschamps, were cruising at 2-0 up at Dijon thanks to early goals from Loic Remy and Benoit Cheyrou.
Brice Jovial gave Dijon some cheer after the break and Benjamin Corgnet levelled only for Alou Diarra to grab victory for Marseille with eight minutes left.
"It's an important victory, the second in a row in the championship," Deschamps told reporters after again reverting to a 4-4-2 system.
"Dijon are not high in the table but they have quality so this gives us confidence and above all the points. We have progressed a little bit."
Marseille and Lyon face Arsenal and Real Madrid respectively in the Champions League in midweek.
In other Saturday games, Girondins Bordeaux stopped the rot with Yoann Gouffran's double in a 2-0 win at bottom side Ajaccio while Stade Brest beat Lorient 3-1 and promoted Evian were in sparkling form with a 3-1 home victory over Auxerre.
