Robert Snodgrass scored the winner two minutes into the second half of a match that saw Spurs fail to register a shot on target until the 81st minute.
And Sherwood bemoaned his side's sluggish start at Carrow Road.
"The result was very disappointing," he said. "The first-half performance was disappointing. Second half wasn't, but (it was) too late and if you don't take your chances, which we clearly had, then you ain't going to win no football matches unfortunately.
"It was a sluggish start and it was a nothing game in the first half. They never did anything, we didn't do anything.
"You were happy to come in at 0-0 and hopefully we could respond, but obviously took a real blow early on in the second half in going behind to a sloppy goal.
"We created a few chances, but unfortunately we couldn't take them."
Asked whether Thursday's trip to the Ukraine to face Dnipro in the UEFA Europa League - a match that Spurs lost 1-0 - had left his players tired, Sherwood stressed: "We were stronger in the second half, so we can't use that as an excuse."
The result leaves Spurs six points adrift of the Premier League's top four, and Sherwood is under no illusions as to how big a blow Sunday's defeat is to his UEFA Champions League aspirations.
"It looks big and it's a real blow to us, but we've got to bounce back," he added.
"It's the first major setback I've had and we're going to need to be spirited to get back and show what we're made of."
