Branislav Ivanovic's 73rd-minute header denied the visitors a draw at Stamford Bridge after Christian Benteke had brought them level by cancelling out an Antonio Luna own goal with an excellent finish.
However, the Serbian defender should arguably not have been on the pitch to score the winner having only received a caution for an apparent elbow on Benteke, while Villa were also denied a penalty for a John Terry handball late on.
And Lambert has expressed disappointment with the official's decisions, believing his side were robbed of a deserved share of the spoils.
"It's easy to come on and have sour grapes because you've lost but it's there for everyone to see," he said. "It's a clear elbow on Christian, it's a sending-off and I think the referee is in the vicinity to do it.
"The penalty is a penalty. Everyone saw that. These are major decisions. You're asked to officiate and you miss two huge moments.
"It was a flying elbow. I saw the penalty, everyone did, how can you miss that magnitude of a decision? It's beyond me."
Despite his frustration, the Scot went on to praise his team's performance, which came just four days after they earned a surprise 3-1 win over Arsenal at the Emirates Stadium.
"I am so proud of my team. I thought they were absolutely outstanding. Petr Cech has kept them in the game," he added.
"There have been some big, big European teams come here over the years but we've given them a right fright, there's no two ways about it.
"The lads are together. They're a really strong group, they are young lads with energy and pace. The character is always there but some of the football? I was really proud of them.
"Now we need to take this form on. We have had two games at Arsenal and Chelsea away. We should have taken more out of them. Then we have Liverpool.
"These are three incredibly hard, tough games but I think everyone who has seen Villa over the last two games will say they're a good side."
Villa face Liverpool in their first Premier League home game of the season on Saturday.
