Cardiff looked to be heading for victory in their first game since owner Vincent Tan's controversial decision to sack Malky Mackay earlier this week, with goals from Jordon Mutch and Fraizer Campbell giving them a 2-0 lead.
However, bottom-placed Sunderland fought back superbly and earned a point through Jack Colback's 95th-minute strike after Steven Fletcher had pulled a goal back seven minutes from time.
Temporary boss Kerslake revealed that Cardiff's players had been left heartbroken by Sunderland's late heroics, and was quick to criticise his side for failing to hang on to their lead.
"The players the devastated in there, we got ourselves into a fabulous position and with the quality we have we expect to see the game out," he said.
"For probably the first 35-40 minutes we blew Sunderland away, but credit to them. Gus (Poyet) has changed his shape and they kept the ball much better.
"When you are 2-0 up and you're under the cosh, that's what we should have done, we should have looked to keep possession better than we did.
"Still, at 2-0 with eight minutes to go we've conceded a poor goal, there is work to be done there."
Ole Gunnar Solksjaer and Sven-Goran Eriksson are both thought to be in contention to take over from Mackay, with Kerslake dismissing suggestions he could assume the role on a permanent basis.
"With all respect, I wouldn't be interested in this job," he added. "I love being a coach, that's where I see myself."
