The two winners will meet in the Wembley semi-finals on April 10/11 when Portsmouth, who beat Birmingham City 2-0 on Saturday, will play either Fulham or Tottenham Hotspur, who drew 0-0 at Craven Cottage.
Stoke, seeking their first semi-final since 1972 having previously knocked out Manchester City and Arsenal, went close after 15 minutes when Hilario punched a Rory Delap long throw straight to Dean Whitehead, whose shot was cleared off the line by Jon Obi Mikel.
However, Chelsea, beaten in their last two games, soon took control and went ahead after 35 minutes with a trademark Frank Lampard shot from the edge of the box.
Defender Andy Wilkinson cleared two goal-bound efforts as Chelsea cranked up the pressure in the second half but he could only deflect John Terry's firm header from a corner that made it 2-0 after 67 minutes.
"It's never easy after a defeat like last week, that's not normal for us," Lampard told ITV. "We've come back ready to put it right and in a very difficult game today I thought we handled it very well."
Reading were the last remaining non-Premier League club in the competition and in the quarter-finals for the first time in 83 years after upsetting Liverpool, Burnley and West Bromwich Albion in previous rounds.
BETTER TEAM
They were by far the better team against a Villa side who lost the League Cup final to Manchester United last Sunday and deserved the lead earned by Shane Long's goals after 27 and 42 minutes.
Villa looked a different team in the second half though and within 10 minutes had turned the game round.
Ashley Young scored after 47 minutes before Carew, a late addition to the starting team after Gabriel Agbonlahor was taken ill in the morning, made it 3-2 with goals after 51 and 57.
Reading came back well and three times went close to an equaliser before Carew's stoppage-time penalty.
Relieved Villa manager Martin O'Neill said he had been happy to reach halftime only 2-0 down.
"After our first-half performance I think most of our players were grateful to get a second chance but just to be back at Wembley is great," he told the club's website.
"Sometimes it is nothing to do with pretty football. Sometimes it is just to do with character, determination and the ability to fight. I think we had lost that in the first half and it came back to us in the second half in great abundance."
