The two winners will meet in the Wembley semi-finals on April 10/11 when Portsmouth, who beat Birmingham City 2-0 on Saturday, will play either Fulham or Tottenham Hotspur, who drew 0-0 at Craven Cottage.
Stoke, seeking their first semi-final since 1972 having previously knocked out Manchester City and Arsenal, went close after 15 minutes when Hilario punched a Rory Delap long throw straight to Dean Whitehead, whose shot was cleared off the line by Jon Obi Mikel.
Defender Andy Wilkinson cleared two goal-bound efforts as Chelsea cranked up the pressure in the second half but he could only deflect John Terry's firm header from a corner that made it 2-0 after 67 minutes.
"It's never easy after a defeat like last week, that's not normal for us," Lampard told ITV. "We've come back ready to put it right and in a very difficult game today I thought we handled it very well."
They were by far the better team against a Villa side who lost the League Cup final to Manchester United last Sunday and deserved the lead earned by Shane Long's goals after 27 and 42 minutes.
Reading came back well and three times went close to an equaliser before Carew's stoppage-time penalty.
Relieved Villa manager Martin O'Neill said he had been happy to reach halftime only 2-0 down.
"After our first-half performance I think most of our players were grateful to get a second chance but just to be back at Wembley is great," he told the club's website.
"Sometimes it is nothing to do with pretty football. Sometimes it is just to do with character, determination and the ability to fight. I think we had lost that in the first half and it came back to us in the second half in great abundance."
