The hosts trailed 2-0 and 3-1 at the King Power Stadium but showed tremendous courage to rally for a first victory over United since 1998, having lost their previous nine Premier League meetings.
A week after seemingly kick-starting their Premier League campaign with a 4-0 thrashing of QPR, United got off to a flying start with early scores via the head of Robin van Persie and an exquisite Angel di Maria scoop.
But Leicester had already taken a point apiece from Arsenal and Everton at home before winning away at Stoke last time out, and displayed their top-flight credentials yet again.
Nigel Pearson's charges responded almost immediately to Di Maria's moment of magic, when their own Argentine record signing Leonardo Ulloa nodded in.
United upped the tempo after the break and capitalised on a sustained period of pressure when Ander Herrera deflected a Di Maria shot into the back of the net.
But the hosts simply would not lie down as a David Nugent penalty and Esteban Cambiasso's low drive put them back on level terms.
Jamie Vardy then kept his cool when put clear and Ulloa put the icing on the cake with another spot-kick after Tyler Blackett saw red, the former Brighton and Hove Albion man scoring his fifth Premier League goal in as many games.
Radamel Falcao made his expected full United debut, replacing the in-form Juan Mata, with captain Wayne Rooney eased back into a slightly deeper role.
Vardy was one of four changes for Leicester alongside former United trainee Danny Drinkwater, Cambiasso and son of Old Trafford legend Peter Schmeichel, goalkeeper Kasper.
Schmeichel was called into early work after Van Persie was fed by Di Maria, but he could do nothing about the Dutchman's deflected header from a superb Falcao cross which put the visitors ahead on 13 minutes.
The lead was doubled in sublime fashion three minutes later when Di Maria linked up with Rooney before the Argentina star lifted a delightful effort over the helpless keeper.
However, United were almost immediately caught cold at the other end when Vardy got to the byline and crossed for Ulloa to head home powerfully to David de Gea's left.
The visitors suffered another setback when Jonny Evans limped off and was replaced by Chris Smalling with less than half an hour gone, but both defences just about managed to hold firm for the rest of a fast and furious first period.
Falcao went close to restoring United's two-goal cushion at the start of the second half when controlling a Rooney pass and firing a half-volley against the crossbar, but it proved a temporary respite for Leicester.
It was Herrera who did claim the third when flicking a Di Maria strike from the edge of the area past a stranded Schmeichel on 57 minutes.
However, Leicester again showed great heart to rally through a Nugent spot kick after Rafael da Silva shoved Vardy in the box and Cambiasso connected sweetly with a shot from just inside the area to level the scores just after the hour.
As United poured forward in search of a winner, they were caught woefully short in defence.
Mata, on as a substitute, was robbed by Ritchie De Laet, whose throughball released Vardy - the former Fleetwood Town man keeping his nerve to slot past De Gea one-on-one.
A mesmerising comeback was complete just four minutes later, more defensive dithering allowed Vardy in on goal again and Blackett could only chop him down in the penalty area, earning the youngster a straight red card and providing Ulloa the chance to cap a stunning outing for Leicester and a first home win in the top flight since May 2004.
