The Swedish champions frustrated their hosts for an hour in Turin on Tuesday before a bit of magic from Kwadwo Asamoah and Carlos Tevez broke the deadlock.
Having received a pass from Tevez, Asamoah produced a sumptuous back heel to return the ball to his team-mate and the former Manchester City striker netted.
A second goal came in the final minute as Tevez again beat Robin Olsen, this time a perfectly-placed free-kick that the goalkeeper could do nothing about.
Hareide sang the praises of his hard-working team and expects them to keep on improving in their new surroundings.
"We did an hour of very good work," said the Malmo coach.
"We laid a good foundation the first 60 minutes, we can improve on it.
"The players were very disciplined.
"[But] there was a man on the pitch that became the difference. It was Carlos Tevez, we never got on top of him."
Malmo's first home game in Group A will see Olympiacos travel to Sweden on October 1.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.