The Villa boss made seven changes to the side that beat Norwich on Saturday but they were undone by some clinical finishing from the Londoners.
The hosts struggled to cope with Tottenham at times and were unable to stop Andre Villas-Boas' side adding three second-half goals to Jermain Defoe's 45th-minute opener - with Paulinho, Nacer Chadli and a second from Defoe confirming the result.
"I didn't think there were four goals in that," he told the BBC. "We had a lot of players out injured and they (Tottenham) were too much for us. But I couldn't fault the lads for effort.
"We gave it a go in the second half and their second goal was a really poor goal from our point of view.
"There are a couple of plus points from individual performances. Nathan Baker is coming back after injury. Marc Albrighton was excellent."
The striker had escaped the attentions of the Dutch defender, but as Vertonghen fell he caught Helenius' shorts and that appeared to hamper his finish.
"That was a big moment (the penalty shout)," Lambert added. "I thought it was a penalty. I don't think anybody could believe it.
"It was a strange decision."
