In a scrappy game devoid of any real scoring chances, N'Doye won it for the Danish champions when he outjumped a Rubin defender to head a free kick into the net off the far post.
Rubin substitute Alan Kasayev had wasted the best chance for the visitors when he misplayed the final pass to an open team mate just a few minutes earlier.
Russian champions Rubin, who shocked Barcelona 2-1 at the Nou Camp on their Champions League debut last year, had more of the play for the first hour but were undone by sloppy passes and could only manage a few shots from long range to trouble Copenhagen keeper Johan Wiland.
Copenhagen coach Stale Solbakken was delighted with his team's performance.
"We played a perfect game," he told a news conference.
"We had a plan that would give us a chance to score from set pieces and we also tried to keep the opposing team away from our own goal. We succeeded in doing both."
Copenhagen eliminated Scandinavian neighbours Rosenborg in the play-off to qualify for the Champions League group stage.
They finished bottom of their group on their only previous appearance in Europe's premier club competition in 2007 but will fancy their chances this time round after clinching the opener.
The Russians clearly missed top striker Alexander Bukharov and midfielder Sergei Semak, whom they sold to league rivals Zenit St Petersburg last month, and they also lost Italian fullback Salvatore Bocchetti to injury in the second half.
Rubin coach Kurban Berdyev said: "It was a real pity that we lost tonight. I don't think we deserve such an outcome.
"We had chances but unfortunately just couldn't take them. Even with five minutes left we had a chance when we had three attackers against only two defenders, but we missed it."
Barcelona crushed Greek side Panathinaikos 5-1 in the other Group D match.
