Scott Rendell scored 10 minutes from time to earn the fifth-tier side a 1-0 win at Norwich's Carrow Road home and leave another indelible mark in the competition's giant-killing history.
Premier League Queens Park Rangers also suffered a humiliating 4-2 exit at the hands of third-tier Milton Keynes Dons to pile on the misery for the struggling London club who are mired deep in a relegation battle.
Drama was in short supply at Old Trafford where Manchester United sauntered past Fulham 4-1 after Ryan Giggs had put the Premier League leaders ahead with a third-minute penalty.
They were joined in the fifth round by champions Manchester City who won another all top-flight clash against Stoke City 1-0 courtesy of a late strike from Pablo Zabaleta.
Second-tier Brighton and Hove Albion pushed Arsenal all the way on the south coast before substitute Theo Walcott scored in the 85th minute to edge his side to a 3-2 win.
Everton's John Heitinga scored in the 90th minute as the Premier League side beat Bolton Wanderers 2-1.
Not since Sutton United beat Coventry City in the third round in January 1989, had a club from outside the top four divisions overcome top-flight opposition in the FA Cup.
Sutton's run that year ended in an 8-0 thumping by Norwich but the Canaries were on the receiving end this time round.
Luton, relegated from the old first division in 1992 and dumped out of the Football League in 2009, lie sixth in the fifth tier of English football.
Norwich had made six changes from the team that lost 5-0 to Liverpool in the Premier League last week in a clear statement of where the competition ranked in the club's priorities.
For 1959 FA Cup runners-up Luton, the occasion was a far cry from their last match, a 0-0 draw with Telford in the Blue Square Bet Premier division.
The gap between the two sides, however, meant little as Luton held off a their opponents before striking late in the second half when Rendell steered home JJ O'Donnell's low cross.
"It's a lot to take in. The players were tremendous today," said Luton manager Paul Buckle, whose team beat second-tier Wolverhampton Wanderers in the third round. "They held a Premier League team for 90-plus minutes.
"It's a huge day for them, for me and for our supporters. Norwich pushed us all the way but we were good for the win in the end. Scotty scored, it's what he does best, but everyone played their part."
NO EXCUSE
The Premier League's bottom club QPR played a second-string team against MK Dons and were comprehensively outplayed by a side 31 places below them in English leagues.
QPR retained only two players from the side that drew 1-1 with West Ham United in the Premier League last Saturday, and were behind after four minutes when Armand Traore put through his own net.
