The Premier League side arguably had the better chances in Wednesday's clash at Villa Park, but Vincelot - on in the 83rd minute for Marvin Bartley - sent the League One side through with a back-post header two minutes from time.
Villa, who have won the competition on five occasions, including the inaugural final in 1961, had the best chance of the first half when Darren Bent - making his first start for Villa since May 2013 - was denied by a smart stop from goalkeeper Gary Woods.
Bent also squandered Villa's best opportunity of the second period when he fired wide when through one-on-one.
Orient had chances too, and they ultimately proved more clinical as Vincelot netted in the dying stages, to send them into the draw for round three.
Close-season signing Joe Cole made his Villa debut, while teenager Jack Grealish also made his first Villa start.
The match began at a lacklustre pace with Orient seemingly content to allow Villa to dictate possession.
Leandro Bacuna proved Villa's most potent weapon and Mathieu Baudry had to be alert to block the forward's dangerous cross into the six-yard box.
It was Bacuna who orchestrated the first shot of the match in the 16th minute when he collected Cole's quick free-kick on the left before shimmying inside and curling the ball towards the far post, where Baudry made the block.
Villa created the best chance of the first half shortly before the half-hour mark.
Bent showed good pace to get beyond the Orient defence and collect Cole's pass into the area, but his shot was deflected wide by Woods' foot.
At the other end, Orient had a good chance when Elliot Omozusi's cross found Dave Mooney unmarked at the back post, but the striker glanced the ball wide.
Kieran Richardson was the first player to try his luck after the break with an ambitious right-footed shot from 30 yards that went wide of the target before Grealish almost broke the deadlock on the hour when his shot was deflected wide.
Orient came into the game as the second half progressed and Villa goalkeeper Shay Given was forced to make a superb stop to push Baudry's powerful header onto the post, before Jay Simpson had the ball in the back of net, only to be denied by the offside flag.
Bent then went close with time running out when he collected Philippe Senderos' throughball but shot wide when one-on-one.
And that miss was punished with just two minutes remaining.
Bradley Pritchard hung up a cross to the back post and Frenchman Vincelot was on hand to brilliantly head home, as Orient claimed a famous win.
