The home side were awarded the spot-kick in the 94th minute when Steven Reid was deemed to have fouled Ramires in the area.
Eden Hazard converted to salvage a 2-2 draw for Jose Mourinho's side, with Clarke claiming referee Andre Marriner was swayed by the home support.
"It's the kind of penalty that can be given when you’ve got a home crowd that shout for everything and put pressure on the referee," he said.
"Referees are human; they make mistakes like everyone else but for us today, the most galling fact is it should have been three points from Stamford Bridge which would have been a fantastic result for us.
"Instead of that it’s just a good result, it’s only a point but the team were tremendous today and I’m really proud of the way they responded in the second half."
"The referee has to be 100 per cent sure it’s a penalty," he continued.
"How he can be 100 per cent sure in that instance is beyond me. It’s not correct – it was a bad decision. I think he went down early before the contact.
"The onus is on the referee to make the right decision –and he didn’t make it."
