Clarke's men were denied three points due to a late penalty at Stamford Bridge in their last outing - a decision referee's chief Mike Riley later apologised for.
Despite this setback, the head coach is eager for his side to move on and focus on their West Midlands derby at The Hawthorns.
"If we can use the frustration to kick on and get the points against Villa then fine," he said, "but the players are all experienced enough to know that you can't dwell on the previous game, you have to look forward to the next one.
"Chelsea's gone and we have to move on. The next game isn't a small game either - it's massive."
The sides are separated by just goal difference in the table and Villa have not beaten West Brom in their last five attempts.
Villa's last victory in the fixture came in December 2010, and Clarke went on to say his side are eagerly awaiting the match.
"It's a big game, we're looking forward to it," he added.
"It's just a shame we didn't have a game quickly after (Chelsea). We've had to sit on it for more than two weeks."
‘Arteta, Alonso, Emery, me… none of us were physical players – we needed the understanding of the game. That probably helped us move into management’: Premier League boss reveals reasons for natural career progression
‘England have the players to win the World Cup – it’ll be tough for Thomas Tuchel to do a bad job, with the squad he has at his disposal’ Former Three Lions winger backs new boss after gentle qualifying draw
