The 1993 European champions lie second in Group F on seven points with one game remaining after substitute Giannis Fetfatzidis's 82nd-minute strike left them with seven points.
Olympiakos will host group winners Arsenal in the last round while Marseille will travel to fourth-placed Borussia Dortmund.
Second-half substitute Fetfatzidis fired home with a low crossed shot after Olympiakos yet again beat the offside trap to give his team a deserved win.
On a mild night in an half-empty Stade Velodrome, Marseille made a shaky start.
In the 10th minute, their defence was caught napping when Olof Mellberg headed home from a free-kick, only for the goal to be ruled out offside.
The warning did not exactly spur the home side into action as Marseille failed to create a clear-cut chance while the Greek side looked dangerous on the break.
It took a couple of brilliant saves by Steve Mandanda to deny Kevin Mirallas and Holebas in quick succession after a swift counter attack left the Marseille defence in duisshambles.
Didier Deschamps's side seemed rejuvenated after the re-start although Olympiakos continued to threaten.
Mirallas once again beat the offside trap but, once again, was denied by Mandanda, who tipped away the Belgium striker's crossed low shot at the hour.
Marseille had their best chance 20 minutes from time when France midfielder Alou Diarra had a downward header blocked by Balasz Megyeri.
Substitutes Andre-Pierre Gignac and Lucho Gonzalez did not help Marseille get any better up front as the hosts intead relied on ugly defending to keep their chances alive.
"We failed to make it simple in the first half. In the second half, they sat back and waited for us and got us on a counter attack," Marseille defender Souleymane Diawara told French TV channel Canal Plus.
"We did not play a European Cup game, we played a friendly game. There was no aggression from our side.
"If we want to qualify we will have to show something else. If we play like tonight, we should not bother to travel to Dortmund."
