The Milan club had squeezed past strugglers Cesena at the weekend and Mazzarri's side left it late to continue their climb back up the Serie A table this time, as Icardi ended his former team's unbeaten start to the season with a goal right at the end.
Under-fire Mazzarri found himself up against one of the reported favourites to replace him should he be sacked, in the shape of Sampdoria coach Sinisa Mihajlovic, but he came out on top against the ex-Inter defender.
The hosts were wasteful in the first half, with Rodrigo Palacio in particular spurning several good chances, but they managed to snatch a dramatic late triumph.
Second-half substitute Zdravko Kuzmanovic was tripped by Alessio Romagnoli with time running out and Icardi, who spent two seasons with Sampdoria, slotted home the resulting spot-kick.
Victory sees Inter continue their march back towards the UEFA Champions League places, while Sampdoria's unbeaten start to the campaign is ended at eight matches, one short of their club record at the beginning of a season.
Daniele Gastaldello was lucky to escape punishment for elbowing former team-mate Icardi in the face early on, before a wayward pass in midfield allowed Mateo Kovacic to play the ball in to Palacio, who inexplicably missed the target from close range.
Palacio was again guilty of wasting a great opportunity in the 13th minute, as the Argentinian latched on to Gary Medel's incisive throughball and poked a shot straight at Sergio Romero from 12 yards.
Inter continued to dictate the tempo as the first half progressed, but they found themselves up against an in-form Romero, who produced an excellent reaction save to keep Icardi's deflected effort out five minutes before the break.
And they were almost made to pay for their wastefulness in the final minute of the half, as Alfred Duncan struck the crossbar after pouncing on a loose ball inside the area.
The start of the second half followed a similar pattern to the first as Palacio spurned another opportunity in the 50th minute, shooting wide from six yards.
Inter's hopes were seemingly dealt a blow a few moments later when Hernanes was forced off with an injury.
Kuzmanovic, the Brazilian's replacement, wasted a glorious chance with 28 minutes remaining, hitting a tame effort straight at Romero when through on goal.
But Inter continued to press and salvaged a dramatic late winner, with Kuzmanovic again central to the action.
He was tripped by Romagnoli when racing through and Icardi fired the resulting penalty into the right-hand side of the net.
‘Ruben Amorim could have waited for Real Madrid and had a better chance to be successful – to have joined Manchester United, he must be convinced in his own ability’ Former Old Trafford coach’s verdict on new boss
‘After Manchester City’s recent form, maybe they’re the underdogs against Manchester United!’ Former Red Devils defender on this weekend’s derby
