Taking charge of his first match after the departure of Portuguese coach Andre Villas-Boas, Sherwood's side lost 2-1 to West Ham at White Hart Lane despite taking the lead with 23 minutes left through Emmanuel Adebayor.
Goals in the last 10 minutes from Matt Jarvis and Modibo Maiga saw Sam Allardyce's side progress to the last four as Tottenham lost back-to-back home games against West Ham for the first time since 1966.
Speaking after the defeat, Sherwood told Sky Sports: "This is a major disappointment both for the boys and for myself and more importantly, for the football club.
"I think we need to defend better as a group. They (the players) need to recognise when to rest on the football.
"(I am) very disappointed. They got their breaks and that was it. We started the game very well.
"I thought we were in command of the game. They stuck at it and bombarded us and got us on the breaks."
Tottenham winger Andros Townsend limped off with a left hamstring injury in the 73rd minute and looks likely to be on the sidelines for the foreseeable future.
Asked if it was a bad injury, Sherwood said: "Looks like it. Looks like he's pulled up with his hamstring.
"It is a big disappointment for him and for everyone."
Sherwood also praised Adebayor after he scored in his first start of the season.
The Togo striker – who has not even been training with Tottenham's first-team at times in this campaign – volleyed home a Jermain Defoe cross to score his first Spurs goal since May.
"I think he did very well," he added.
"The unfortunate thing is that him and Danny Rose had to come off the field because (if I left them on) it would risk their fitness for a few weeks.
"I tried to drag it out as long as I could."
