The former Manchester United and Real Madrid player met compatriot Luis Gonzalez's cross with a first-time shot following a quickly-taken throw-in after being left unmarked in the penalty area in the 69th minute.
The win pulled the French team level on three points from three games with the Swiss champions, who rarely looked like producing a repeat of their stunning 1-0 win at AC Milan last month as they slumped to their second home defeat in the group.
"I didn't see a great match," said Marseille coach Didier Deschamps.
"We started very passively until the last few minutes of the first half, there was a lot of physical play and impact and Zurich didn't have any chances.
"We can do better."
"Maybe we're too young for the Champions League," he said. "We were too naive at Heinze's goal."
Marseille, who lost their previous six away games in European competition, had the ball in the net in the third minute when Brandao headed home a rebound after Johnny Leoni failed to hold Benoit Cheyrou's free kick.
Leoni produced the first half's high point when he denied Brandao with a stunning point-blank save. Gonzalez also threatened with a 25-metre shot and then tested Leoni with a header.
Mandanda finally saw some action immediately after Heinze's goal when he tipped Gajic's vicious 35-metre free kick onto the crossbar after the ball flew through a crowd of legs.
