Schalke 04's 3-0 win over Olympique Lyon in the other Group B tie means Benfica cannot reach the knockout stage.
Hapoel coach Eli Guttman said it was an "evening of pride" for his club after they secured victory with two goals by Eran Zahavi and another from Douglas da Silva.
The hosts had far less possession but maintained their resolve and found the finishing touches which Benfica clearly lacked.
In answer to a question if before the match he thought Hapoel could beat Benfica 3-0, Guttman said: "Never... if you had told me this before the match I would say you are drunk, but football is football, everything is possible and we showed it today."
He added that it was the players' way to give their "wonderful crowd what they deserved to see."
Zahavi made the breakthrough after 24 minutes when he headed in a free-kick from playmaker Gil Vermouth.
Da Silva doubled the lead in the 74th minute with a shot from close range following a Hapoel corner, their first of the match. Benfica, by contrast, had 20 corners.
Zahavi secured Hapoel's first win in their debut season in the competition when he added his second goal in injury time after a swift counter attack.
Schalke top the group on 10 points with Lyon on nine. Benfica have six points with one match to play while Hapoel, on four points, retain a slim chance of finishing in third place if they beat Lyon next week in France and Benfica lose to Schalke.
Benfica coach Jorge Jesus said his side were unlucky and once they fell behind their chance of winning faded.
"We controlled the game, we had seven or eight chances in the first half but once Hapoel scored it became increasingly difficult for us," Jesus said.
