Ryan Flynn's 81st-minute strike on Saturday resulted in Villa exiting the competition at this stage for the first time since 2007-08 and piles the pressure on Paul Lambert at the end of a week in which he admitted he would rather avoid the distraction of the FA Cup to ensure Premier League survival.
A draw looked likely after Nicklas Helenius scored his first Villa goal to cancel out Jamie Murphy's opener, but Flynn sealed victory for Nigel Clough's side, who are 18th in League One.
Like Villa, Leeds United will not be in the hat for the fourth-round draw on Sunday after Rochdale produced an impressive 2-0 win at Spotland.
Scott Hogan's fifth goal in four games put the League Two side ahead just before half-time and Ian Henderson added a second six minutes from time.
Holders Wigan Athletic face a replay after being held 3-3 by MK Dons at the DW Stadium, while Manchester City could not get past Blackburn Rovers and will meet against at the Etihad Stadium following their 1-1 draw.
Arsenal defeated fierce local rivals Tottenham 2-0 at the Emirates Stadium to progress. Santi Cazorla and Tomas Rosicky confirmed the victory with goals in either half to extend the side's unbeaten home run against their north London neighbours to four games in all competitions.
Roberto Martinez, who won the trophy with Wigan last season, saw his Everton side convincingly beat QPR 4-0 at Goodison Park with Nikica Jelavic grabbing a brace and missing a penalty, while Ole Gunnar Solskjaer claimed a 2-1 win at Newcastle United in his first game in charge of Cardiff City.
Stoke City saw off the threat of Championship leaders Leicester City 2-1, Southampton survived a late scare to beat Burnley 4-3 and Tony Pulis' Crystal Palace defeated West Brom 2-0 thanks to goals from Dwight Gayle and Marouane Chamakh.
Coventry City came from a goal behind against Barnsley to seal a 2-1 victory at Oakwell, with another Championship side in Doncaster Rovers also knocked out after being beaten 3-2 by Stevenage.
Two of the three non-league sides in the third round will face replays after earning draws at home. Macclesfield Town had Steve Williams to thank as he cancelled out Reda Johnson's opener for Sheffield Wednesday to seal a 1-1 draw at Moss Rose, while Kidderminster held Peterborough United 0-0 at Aggborough.
Grimsby Town, the third non-league outfit, were denied a memorable victory over Huddersfield Town as Mark Robins' side scored twice in the last five minutes to win 3-2. Grimsby had led thanks to Ross Hannah and Craig Disley either side of Oliver Norwood netting for Huddersfield, but Martin Paterson's fourth goal in as many games and an Aswad Thomas own goal saw them exit the competition.
Bolton Wanderers, Brighton and Hove Albion and Yeovil Town also booked their places in the next round with victories over Blackpool, Reading and Leyton Orient respectively, but Millwall went down 4-1 at Southend United.
Elsewhere, Bristol City held Championship side Watford 1-1 at Ashton Gate, Kevin Davies earned Preston North End a 1-1 draw and a replay against Ipswich Town and Robert Snodgrass cancelled out Darren Bent's 40th-minute strike as Norwich drew 1-1 with Fulham.
Three FA Cup games were called off due to waterlogged pitches so Bournemouth and Burton Albion, Charlton Athletic and Oxford United and the second-round replay between Crawley Town and Bristol Rovers will all be rearranged.
