Ricky Alvarez put the visitors ahead in the 16th minute, firing home from six yards after gathering Rodrigo Palacio's knockdown but Mazzarri's side were soon pegged back as German Denis levelled matters just nine minutes later.
It was Denis' sixth goal in three matches against Inter after the forward's five goals in Atalanta's double over their opponents last season.
After the game, Mazzarri insisted Inter should have won, lamenting their wastefulness in front of goal.
"In my view we deserved to win," the 52-year-old told Sky Sport Italia.
"If you look at the scoring opportunities, then we created more than Atalanta this evening, especially in the second half.
"We need to be more clinical and sharp in attack.
"We also have to improve our attitude and score more goals in this period that we're missing Diego Milito.
"We were naive to give away a goal when in front. We have many aspects to improve upon in order to battle for the top spots."
Despite his frustrations, Mazzarri reserved praise for his opponents and specifically their goalscorer.
"Atalanta are a strong side and I complimented (Atalanta coach) Stefano Colantuono," the former Napoli head coach said.
"Denis is one of the best centre forwards in Italy. He didn't play that much for me at Napoli but when he did he was always ready. We shouldn't have let that cross find its way to him."
Inter goalscorer Alvarez, meanwhile, was frustrated by his side's inability to push home their advantage once in front.
"We don't keep hold of our leads," he said.
"We need to be smarter. Once we went in front, we were unable to keep hold of the lead. That is what we were missing against Torino and Cagliari too.
"In every game we take the lead, but don't tend to keep hold of it. Anyone who wants to fight for the Scudetto has to be able to do that and so far we've failed.
"Today we leave with a bitter taste in our mouths."
