Going into the game, Lille knew only a win would do to secure qualification and Trabzonspor were seeking a victory while also being aware that a point would be enough if CSKA Moscow did not win at group winners Inter in the other match.
Hosts Lille came rapidly out of the traps with midfielder Eden Hazard having three good chances but the French champions wilted after the break against a dogged defence.
Trabzonspor, now bound for the Europa League, wrongly thought they were going through to the Champions League first knockout round even after a decisive late goal gave CSKA victory at Inter but the news eventually filtered through.
Lille boss Rudi Garcia made two defensive substitutions despite needing a victory, taking attacking midfielder Joe Cole off on 69 minutes and replacing a fullback with another as frustrated fans inside the Metropole shook their heads.
Ludovic Obraniak, who came on for Cole as his side slipped to bottom of the group, told reporters: "We did everthing we could tonight. But it was not enough. We still have our national title to defend but we are all disappointed.
"The Champions League is a prestigious competition. We wanted to go through. It was just impossible tonight."
INTRICATE PLAY
The French champions had started brightly with Belgian Hazard having a superb shot saved after a strong run, before firing just past the post after more intricate play and then blasting wide.
Striker Moussa Sow, top scorer in Ligue 1 last term, also had a couple of sniffs while Dimitri Payet and Florent Balmont also went close but Lille failed to make their dominance pay in the first half.
Trabzonspor were not in the Champions League originally but gained entry after Fenerbahce were kicked out following a domestic match-fixing scandal, with a probe also now investigating Trabzon among others.
The Turks won their opener at stunned 2010 champions Inter, who recovered to go through and top the group, but Senol Gunes's side have continued to impress and were only undone by their inability to up the tempo in Lille once CSKA scored a winner.
They were mainly forced to defend in north eastern France but Burak Yilmaz shot just over in a rare foray forward in the first half.
