Ghana forward Ayew struck in each half and France striker Loic Remy was also on target in a lively game to put Marseille firmly in command of Group F with six points from two games.
France keeper Steve Mandanda was also instrumental in the former champions' win, making a series of superb saves throughout.
Marseille's night finished on a sour note when substitute Jordan Ayew was sent off in stoppage-time, picking up a second yellow card for diving.
"Winning a Champions League game does not happen that often," coach Didier Deschamps, who was in the Juventus team that lost 3-1 in the Champions League final against Dortmund in 1997, told a news conference.
"But we were not unlucky tonight, they did not deserve to lose with such a big margin. It was not perfect but we've got six points."
"We now have two games to play against Arsenal, who are the best team in the Group, and we need to take points against them," said Deschamps.
"We could have equalised on several occasions. In the first half, but also in the beginning of the second half," Klopp told a news conference.
"We made several mistakes and we paid for them. Sometimes you just have to kick the ball away.
"Even when we enjoy possession, we're not patient enough. We played well but we still lost 3-0."
Deschamps's side doubled the lead in the 62nd minute against the run of play when a Mats Hummels header landed into the path of Remy, who volleyed home from inside the box.
"It was another competition, but it's a morale-boosting performance," midfielder Mathieu Valbuena told reporters.
"We now have to do the same in Ligue 1 and it starts against Brest on Sunday," added holding midfielder Alou Diarra.
