The AC Milan defender, making his first competitive appearance in seven months following knee surgery, headed in Angelo Palombo's cross-come-shot on 56 minutes in Tbilisi before miskicking a clearance which also flew into the net.
"My team did well and we showed our heart. I'm so sorry, it's my fault. It hurts me," he told Italian reporters late on Saturday. "I don't know what to say."
Group leaders Italy host Bulgaria on Wednesday where a win would put the world champions four points clear of second-placed Ireland and needing just a point against the Irish in October to reach the finals in South Africa next year.
However, Marcello Lippi's lacklustre side have now not scored in four matches if Kaladze's goals are discounted.
Midfielder Andrea Pirlo had another game to forget while forwards Fabio Quagliarella and Vincenzo Iaquinta summed up their growing problems by both striking the post in injury time.
Kaladze said the Azzurri were still creating chances, however, and expected them to find their scoring boots again soon.
"They are only missing putting the ball away when there is a chance. Quagliarella was unlucky, that's all that's missing," he said.
