Rodgers' men slumped to a 2-1 defeat at Wembley last Sunday, with Villa turning the game around to reach the final after Philippe Coutinho had put Liverpool ahead.
That result ended hopes of outgoing captain Steven Gerrard bidding farewell by lifting the FA Cup in his final appearance before joining LA Galaxy and has upped the pressure on Rodgers, with Liverpool outsiders for a UEFA Champions League spot.
Liverpool are seven points adrift of fourth-placed Manchester City, although they do have a game in hand.
But Carragher told Sky Sports: "Is he the right man? I think so. I don't think your whole future should be determined by one game. It was difficult with the players that came in and I still don't think many of them have bedded in yet.
"Brendan, the players and the staff will be looking to next season and thinking they will go one step further, winning a trophy maybe and getting back into the top four.
"That's his job. He has to identify where in the team they need to improve; the squad is there and they did a lot of that last summer.
"The signings have to be right and if they are it will make a massive difference. If they're not they'll be back to where they are this season."
Asked about the need for Liverpool to tie vice-captain Jordan Henderson down to a new contract, Carragher added: "I think he's vitally important.
"He's an England player and although he's not a local player, the way he performs for Liverpool shows you the passion that he's got.
"He's a regular for England and without Steven Gerrard he's taken on the role of the captaincy this season. The manager has said it isn't set in stone that he will get it next season but I'd be very surprised if he wasn't.
"His record as captain is very good and he's a player who has grown into playing for Liverpool after a difficult start."
