In a scrappy game devoid of any real scoring chances, N'Doye won it for the Danish champions when he outjumped a Rubin defender to head a free kick into the net off the far post.
Rubin substitute Alan Kasayev had wasted the best chance for the visitors when he misplayed the final pass to an open team mate just a few minutes earlier.
Copenhagen coach Stale Solbakken was delighted with his team's performance.
"We played a perfect game," he told a news conference.
"We had a plan that would give us a chance to score from set pieces and we also tried to keep the opposing team away from our own goal. We succeeded in doing both."
They finished bottom of their group on their only previous appearance in Europe's premier club competition in 2007 but will fancy their chances this time round after clinching the opener.
Rubin coach Kurban Berdyev said: "It was a real pity that we lost tonight. I don't think we deserve such an outcome.
"We had chances but unfortunately just couldn't take them. Even with five minutes left we had a chance when we had three attackers against only two defenders, but we missed it."
