Long leaves Liverpool season in tatters
The Championship strugglers emulated League One Leeds United's epic 1-0 victory at Manchester United to set up a fourth-round tie at Premier League Burnley.
It was the first time since 1959 that both Manchester United and Liverpool have failed to reach the fourth round and Liverpool's season is in grave danger of fizzling out.
Rafa Benitez's men are 12 points behind leaders Chelsea in the Premier League title race and have been eliminated from the Champions League, League Cup and FA Cup, leaving the Europa League as their only realistic trophy target.
In the night's other replay, Championship leaders Newcastle United beat Plymouth Argyle 3-0 at St James' Park with a hat-trick from Danish striker Peter Lovenkrands to set up a trip to fellow Championship promotion-chasers West Bromwich Albion.
Reading's victory was one of the great giant-killing feats of recent years and took their minds off their relegation fight from the Championship, just two years after they dropped out of the Premier League.
Reading looked to be heading out but they equalised with a penalty four minutes into stoppage time from Gylfi Sigurdsson after Yossi Benayoun sent substitute Shane Long sprawling in the penalty area.
Sigurdsson kept his cool and sent Liverpool goalkeeper Diego Cavalieri the wrong way.
Long glanced home Reading's winner in the 10th minute of extra time to secure their first ever victory at Anfield.
Liverpool had taken the lead in stoppage time at the end of the first half when Ryan Bertrand deflected a Steven Gerrard cross into his own net.
Reading, however, played the better football for most of the match and Jobi McAnuff went close in the second half to scoring one of the most memorable FA Cup goals for years.
Picking up the ball in his own half, he jinked his way through five Liverpool players before finding himself on his own in space in the Liverpool penalty area.
However, with only Cavalieri to beat he scuffed his shot narrowly wide of the post.
Liverpool, FA Cup winners seven times, rarely resembled any of their great teams of the past and finished the match without talismanic captain Steven Gerrard and top scorer Fernando Torres who were both injured and are doubtful for Saturday's visit to Stoke City.