Top goalscorer Andre-Pierre Gignac grabbed a last-gasp winner as Marseille won 2-1 at Stade Michel d'Ornano - extending their lead to five points in the process.
All three goals came in an action-packed final 16 minutes, with Marseille's Alaixys Romao breaking the deadlock in the 74th minute.
After having a goal ruled out for offside, Caen levelled proceedings through Yroundu Musavu-King with six minutes remaining.
However, Marseille were not to be denied a seventh consecutive win thanks to Gignac, who netted his eighth goal of the campaign in the third minute of stoppage time.
"The equaliser we conceded near the end placed doubt in the minds but the team believed in our chances right up until the final whistle," Bielsa told reporters post-game.
"We attacked well all game and created 15 chances. It was a tough win as we allowed them to approach our goal and have chances they didn't deserve given the way the match went."
While Marseille were far from their best, midfielder Dimitri Payet was just happy to collect all three points before the international break.
"It was not our best game but we won, and it's important to win in this kind of meeting," said Payet.
"It is a beautiful line of victories, just before the break, it feels good."
