An own goal from Nikolay Bodurov sealed Croatia's second win from as many Group H matches in Euro 2016 qualifying.
Kovac is under pressure following Croatia's exit at the group stages of the 2014 World Cup but his side have made an excellent start to their qualifying campaign.
And the nature of Croatia's goal pleased Kovac, who said: "We scored from a corner – we've been practising set-pieces in the past few days and it paid off.
"I have players in this team that are capable of producing killer passes, but that didn't happen this time.
"Even so, I am happy with the three points.
"I don't know whether I will come in for less criticism now.
"The most important thing is that we performed well and that is a credit to the players."
Croatia defender Vedran Corluka felt his side were excellent at the back as they claimed a difficult road win.
"I think we deserved the win," Corluka said.
"We played extremely well in defence. We were totally focused and careful and did not allow Bulgaria to create clear chances.
"The victory is very important as we now have six points from the first two games. Sofia is a tough place to go. I am sure other teams in the group will have also tough visits here."
Bulgaria coach Luboslav Penev is tempted to swing the axe after the disappointing defeat and said: "I accept the defeat, but not the manner of it.
"Key players did not show their true qualities. I did not expect to lose, but we could not win the match with that kind of performance.
"Maybe there will be changes for the next game."
