The home side opened the scoring after just seven minutes when Ivan Marcano crossed from the left and found Jose Holebas, who outmuscled two defenders and planted a firm header past keeper Roman Weidenfeller.
Robert Lewandowski equalised but goals from Rafik Djebbour and Francois Modesto sealed victory for the Greek champions.
"We knew that if we were to lose today, we would be out of Europe," said Olympiakos coach Ernesto Valverde. "Our players had the experience to handle the situation better as the game wore on.
"Dortmund played a fast game and pressed high up the pitch but we were able to take control of the game in the second half."
Olympiakos and Borussia traded chances before Polish striker Lewandowski, who had earlier seen a goal ruled out for offside, teed up a volley on the edge of the area and fired past Franco Costanzo to equalise for the visitors in the 26th minute.
Kevin Mirallas missed a couple of good chances for the home side before his good work on the right led to Djebbour putting Olympiakos back in front. The Algerian received the ball in the penalty area, evaded his marker and fired home in the 40th minute.
Four minutes later, Djebbour had the chance to extend the lead but he volleyed the ball into the ground and saw his shot skew the wrong side of the post.
Olympiakos continued to create chances in the second half. Defender Modesto headed straight at Weidenfeller in the 62nd minute when unmarked at a free-kick. Mirallas missed another great opportunity three minutes later as he failed to finish off a rapid counter-attack.
Modesto made amends for his miss on 77 minutes, when he flicked an Ariel Ibagaza free-kick past Weidenfeller to seal the three points for the home side.
Having lost their first two games in Group F, this was a more convincing performance from the Greek champions who will travel to Dortmund for their next game on November 1 hoping for a similar performance.
Dortmund coach Jurgen Klopp blamed his side's inexperience.
"Olympiakos is a good team and I wasn't surprised by their quality but we let them go in at half-time with the lead," said Klopp.
"My team's lack of experience means that we are not yet fully aware that we need to be totally focused in these games."
