Antonio Di Natale scored the winner for Udinese with a 68th-minute penalty, while the visitors had Cristian Zapata and second-half substitute Kevin-Prince Boateng dismissed in their third defeat in four Serie A games.
Inter Milan also went down to a shock defeat, beaten 2-0 at home by lowly Siena with Simone Vergassalo and Francesco Valiani scoring in the last 15 minutes.
Neither Milan nor Inter have won a game at San Siro this season in any competition in eight attempts between them.
Napoli, Sampdoria and Lazio all lost their 100 percent records, leaving Juventus as isolated leaders with 12 points from four games after their 2-0 win over Chievo on Saturday.
Napoli, who have 10 points, were held 0-0 at Catania and Sampdoria were held 1-1 at home by Torino in a match between two promoted teams.
Sampdoria have nine points, having had one deducted over their involvement in the Calcioscommesse match-fixing scandal.
Lazio slumped to a shock defeat 1-0 defeat at home to Genoa as they dominated the game, missed a hatful of chances and were caught by a breakaway goal scored by Marco Boriello in the 79th minute.
Palermo's latest coach Gian Piero Gasperini got off to a losing start as his side lost 1-0 at Atalanta and promoted Pescara picked up their first point with a 1-1 draw at Bologna.
The action took place against the backdrop of the farcical events in Sardinia where the Cagliari city government called off the Roma match after the home club defied its previous decision to stage the game behind closed doors.
On Saturday, Cagliari slammed "bureaucratic difficulties and the collective disinterest of the authorities" and invited fans to attend the game.
Cagliari have moved to the Is Arenas on the outskirts of the Sardinian capital but the local authorities ruled earlier this week that the stadium was not ready to host Serie A games despite recent improvements.
Milan were facing opponents who, like themselves, are struggling to rebuild after selling their top players during the close season.
Udinese, who had not previously won a game this season in any competition, went ahead five-minutes before the break when Swedish striker Mathias Ranegia scored from close range following a free-kick.
Boateng, rested from the starting line-up, came on in the 52nd minute and had an immediate impact as he was involved a move which ended with Stephan El Shaaraway rifling the equaliser from 30 metres.
Milan, who channeled too much energy into arguing with the referee and wrangling with their opponents, failed to build on that and fell behind again in the 68th minute.
Zapata tripped Ranegia, picking up his second yellow card in three minutes, and 35-year-old Di Natale converted from the spot.
Milan's hopes of fighting back ended when Boateng was booked for fouls on Giampiero Pinzi and Roberto Pereyra
