Mike Williamson was the fall guy for Newcastle United as his 76th-minute own goal meant Championship Brighton and Hove Albion knocked out the Premier League side 1-0 in the late kick-off.
Fourth-tier Crawley Town stunned second-division Hull City by the same score in the only other shock, while Chelsea also beat Queens Park Rangers 1-0.
United left-back Evra was given a hostile Anfield reception after the Frenchman's involvement in a spat with Liverpool's Luis Suarez in October, which led to the Uruguayan forward's current eight-match ban for racial abuse.
Kenny Dalglish's side, still missing the verve offered by Suarez, took the lead on 21 minutes when Daniel Agger headed in a corner that struggling United goalkeeper David de Gea failed to collect after he was crowded out by the hosts.
United, without the injured Wayne Rooney, hit back through a crisp strike inside the near post from Park Ji-sung on 39 minutes and the visitors had other good chances, with Antonio Valencia hitting the post.
Liverpool had the last laugh, though, when Andy Carroll, widely criticised for his form since his move from Newcastle last year, flicked on the ball to leave Evra out of position and Kuyt slammed it in when De Gea could again have done better.
Asked if Liverpool had merited victory, captain Steven Gerrard told ITV Sport: "Not on possession, probably not on general play; we've got to give Manchester United credit, they controlled large periods of the game today, but the most important thing was the result and we got there in the end."
United boss Sir Alex Ferguson, whose side beat Premier league title rivals Manchester City in the third round but are now out of the FA Cup as well as the Champions League, agreed.
"I don't know how we lost it, it was a really good performance by us," he said.
NO HANDSHAKES
Liverpool were woeful in a league defeat at struggling Bolton Wanderers last weekend but eliminated Premier League leaders City on Wednesday to reach the League Cup final and have now beaten bitter rivals and league champions United.
Dutchman Kuyt's strike sent Anfield into raptures and prompted more hopes in the city that Dalglish can bring back the glory days from his previous spell in charge.
The goal also avoided a replay at Old Trafford next month, which Suarez would have been eligible to play in.
Racism accusations were also the backdrop to Chelsea's win at Queens Park Rangers, where John Terry, in court next week after being charged over comments allegedly made to QPR's Anton Ferdinand in October, was kept away from his fellow defender as the clubs decided not to shake hands before kick-off.
Like Evra, Terry was constantly heckled by the home fans whose wrath switched to the referee on 62 minutes when he awarded a soft penalty to Chelsea after Clint Hill was adjudged to have pushed Daniel Sturridge. Juan Mata converted the spot-kick.
