Defender Souleymane Diawara headed in three minutes into the second half to hand the Ligue 1 champions all three points.
The victory left Didier Deschamps's side third in Group F with three points from as many games, six points adrift of Chelsea who beat second-placed Spartak Moscow 2-0 in Russia.
Spartak have six points, Zilina have nonee.
"It was not an easy game but we did our job to claim this victory," Diawara, who otherwise produced a mediocre performance in defence, told reporters.
"We broke the deadlock on a set piece once again before struggling towards the end of the game."
The Slovakian league champions made the better start and were denied a penalty in the third minute when Diawara brought down Momodou Ceesay.
BOOED
Marseille's Andre-Pierre Gignac was then twice denied by keeper Martin Dubravka as the hosts put the pressure on at a windswept Stade Velodrome.
Mathieu Valbuena was his usual speedy self on the right flank and perfectly set up Andre Ayew midway through the first half only for the Ghana forward to waste his volley from just inside the box.
Marseille should have gone ahead on the stroke of halftime but Brandao somehow managed to miss the target.
Gignac headed across the face of the goal and the Brazilian striker shot over from a few inches.
Diawara eventually broke through when he nodded in a Lucho Gonzalez corner, Dubravka failing to block the ball and diverting it into his own net.
Tomas Majtan came close to an equaliser shortly after the hour when he outpaced Cesar Azpilicueta but keeper Steve Mandanda pulled off a super save to parry the Zilina striker's effort from inside the area.
Brandao was booed off the pitch as he was replaced by France forward Loic Remy, who quickly went into action and could become a regular starter at the expense of the out of form Brazilian.
"We now have to go and get a good result in Moscow but it will not be easy, especially because of the (cold) weather in Russia," said Diawara.
