Conte was made to sweat over his 100 per cent start to life in charge of the national side, despite the hosts dominating proceedings at the Stadio Renzo Barbera in Palermo.
Defender Giorgio Chiellini ultimately stole the show, as he scored all three goals, including a late winner.
Having put Italy in front shortly before the interval, the Juventus defender looked set to undo his good work with an own goal 13 minutes from time.
But while the experienced centre-back eventually proved the match-winner by scoring again at the right end with eight minutes left, Conte was left frustrated as Italy made hard work of the Group H points.
"In all honesty, I have little to complain about, other than the fact the players weren't determined enough in front of goal," the Italy coach told Rai Sport.
"Azerbaijan didn't have a shot other than the goal and scored out of nowhere with that bundle over the line.
"We reacted well, proving this team has what it takes. We need to be more clinical, more determined and we wasted too much tonight.
"Azerbaijan came here to stop us playing and hope to score from a set-play. Their plan was actually working [but] we managed to scrape a small win, but deserved much more."
