Manchester City crushed League One Notts County 5-0 in a fourth-round replay and Bolton Wanderers won 1-0 at Fulham in an all Premier League fifth-round tie.
The result in east London left Orient fans celebrating as if they had won the game and dreaming of finishing the job in the replay at Arsenal's Emirates Stadium to earn a quarter-final meeting with Manchester United at Old Trafford.
Tehoue, a pacy but erratic 26-year-old French striker who is often used as a late substitute, squeezed past defenders Kieran Gibbs and teenage debutant Ignasi Miguel on the edge of the penalty area and smashed the ball through the legs of Arsenal goalkeeper Manuel Almunia.
Until then Arsenal, who made 10 changes from the side that beat Barcelona in the Champions League on Wednesday, looked comfortable after taking the lead through a 53rd-minute Tomas Rosicky header.
Orient had been largely outplayed and outpassed as Arsenal appeared to be coasting to victory and a clash with United in the last eight.
But Tehoue's goal means they will replay at the Emirates on March 2 with the money raised set to provide a financial lifeline for the League One club, 53 places below Arsenal in the ladder.
VEGAS TRIP
Orient chairman Barry Hearn promised his squad a trip to Las Vegas if they avoided defeat and laughed afterwards that his manager Russell Slade had out-thought Arsenal's astute boss Arsene Wenger by leaving it late to bring on Tehoue.
"The best thing for us was that it was 1-0 with only a couple of minutes to play because Arsene is not going to put on any of his main line from the bench, because he is trying to nick it, get away with a 1-0 win," Hearn said.
"If we'd have equalised in the 70th minute, on would have come (Cesc) Fabregas and all the rest of them, and we'd have been in trouble. "But Arsene got a little bit complacent and our Frenchman put the boot in," he joked.
Wenger told ESPN that Tehoue had made a difference.
"He scored a great goal but we had the right attitude, we were focused and they took advantage of one of our few mistakes," Wenger said.
"It was difficult for us to create more chances and credit to them for they defended well."
CITY THROUGH
Manchester City won in style against Notts County with Carlos Tevez, Edin Dzeko and Micah Richards scoring late goals to add to two Patrick Vieira headers.
City, transformed by the money of Abu Dhabi billionaire owner Sheikh Mansour and seeking their first major trophy in 35 years, face Aston Villa in the fifth round.
Bolton maintained their hopes of a first FA Cup final appearance since they last won the competition in 1958 by ending Fulham's interest at Craven Cottage.
Ivan Klasnic scored the only goal after 19 minutes, and the highlight of the day for the home s
