Milan looked set to boost their chances of securing a place in the UEFA Europa League, and salvaging something from a largely disappointing campaign, when they took the lead early in the second half.
A Gianpaolo Bellini own goal put Milan in front in the 51st minute, but the hosts drew level before claiming victory in sensational fashion.
Top scorer German Denis equalised from the penalty spot for Atalanta with 22 minutes remaining before substitute Brienza had the final say.
With virtually the last kick of the game, the 34-year-old curled a wonderful left-footed shot into the top-left corner from 30 yards, heaping more pressure on Seedorf.
Sadly, matters on the field were marred by a banana being thrown onto the pitch in the direction of Milan defender Kevin Constant, an action that was met with sarcastic applause from team-mates Philippe Mexes and Nigel de Jong.
Milan made three changes to the side that beat city rivals Inter last weekend, Marco Amelia, Sulley Muntari and Keisuke Honda coming in for Christian Abbiati, who was suspended, Andrea Poli and Adel Taraabt.
Chances proved few and far between in a first half dictated for long periods by Milan, although the hosts may have taken the lead had Carlos Carmona been able to provide a better finish midway through the first half. The Chilean's weak shot was comfortably saved by Amelia.
Atalanta again went close in the 29th minute as Denis fired wide from the edge of the area, before Maxi Moralez missed the target with a header from Luca Cigarini's free-kick.
However, the best opportunity of the first half came five minutes before the interval as Atalanta goalkeeper Andrea Consigli tipped Mario Balotelli's curling free-kick against the crossbar.
Consigli was less convincing moments later when Balotelli fired in another swerving free-kick that he could only parry, with Honda heading wide on the follow-up.
Stephan El Shaarawy came on to replace Honda at the start of the second half, the forward returning from injury to make his first appearance of 2014.
Kaka then flashed a header over the bar before Seedorf's men broke the deadlock, albeit in somewhat fortuitous fashion.
Muntari broke down the left and aimed a low cross towards Balotelli, who appeared to be in an offside position. In attempting to prevent the ball from reaching Balotelli, Bellini turned the ball into his own net.
However, the visitors failed to take control of the contest after going ahead and Atalanta levelled when Denis sent Amelia the wrong way with a confident penalty after Carmona had been tripped by Constant.
Balotelli was thwarted by the woodwork again, hammering an effort against Consigli's right-hand post as Milan pressed for a winner.
Yet there proved to be one final twist in the tale as Brienza earned Atalanta three points with a brilliant first-time shot.
