Chilean winger Mauricio Isla scored the only goal from a lightning counter-attack in the 73rd minute to give Udinese 27 points from 13 matches, the same as Milan who have a better goal difference.
In a pulsating finale Javier Zanetti (Inter) and Damiano Ferronetti (Udinese) were dismissed while Antonio Di Natale and Giampaolo Pazzini missed penalties for each side.
"It's not that common for Udinese to win at the San Siro," visiting coach Francesco Guidolin told Sky Sports.
"We are adding up the points and to be top is marvellous. We must live in the present though."
"We have to be quiet now and work hard," said coach Claudio Ranieri. "If we are far off the title, there are still other prizes to be had this season.
"Without crying we have to fight and see where we are by the end of the year. This should have been the game to turn things around."
It was Udinese who dominated the early stages though with Romanian midfielder Gabriel Torje warming Julio Cesar's hands after 12 minutes with a powerful shot from distance.
In contrast to the visitors, Inter's approach play was laboured but from a flowing move in the 16th minute Milito lifted the ball over the bar after finding space on the edge of the box.
The Argentine striker cracked a powerful free-kick just over 13 minutes later and Thiago Motta then swapped passes with Milito to create the best chance of the first half but the midfielder's shot was too close to keeper Samir Handanovic.
Inter skipper Zanetti received his marching orders with four minutes to go after collecting a second booking for bringing down Kwadwo Asamoah but Di Natale's penalty was turned aside by Julio Cesar.
At the San Paolo stadium, Napoli broke Lecce's resistance with three goals in a 15-minute spell before half-time.
