The home side were awarded the spot-kick in the 94th minute when Steven Reid was deemed to have fouled Ramires in the area.
Eden Hazard converted to salvage a 2-2 draw for Jose Mourinho's side, with Clarke claiming referee Andre Marriner was swayed by the home support.
"It's the kind of penalty that can be given when you’ve got a home crowd that shout for everything and put pressure on the referee," he said.
"Referees are human; they make mistakes like everyone else but for us today, the most galling fact is it should have been three points from Stamford Bridge which would have been a fantastic result for us.
"Instead of that it’s just a good result, it’s only a point but the team were tremendous today and I’m really proud of the way they responded in the second half."
Clarke was angered by the nature of the penalty as well as its timing, as his team were denied back to back Premier League wins
"The referee has to be 100 per cent sure it’s a penalty," he continued.
"How he can be 100 per cent sure in that instance is beyond me. It’s not correct – it was a bad decision. I think he went down early before the contact.
"The onus is on the referee to make the right decision –and he didn’t make it."
‘Maybe I’ll be signed by Wrexham, who knows? English football would suit me – if an offer came, I wouldn’t think twice, I’d go immediately’ Euro 2024 cult hero reveals his UK dream
‘Managing Leeds? It was an option that appeared, but it wasn’t the right timing. I decided it wasn’t a good idea to leave the club I was at mid-season’: Premier League boss admits to turning down opportunity to replace Jesse Marsch in 2023
