Weir, 43, was given his first managerial job in June when he signed a three-year deal at Bramall Lane.
The former Everton and Rangers defender won his first game in charge of the club, 2-1 against Notts County, but the wheels fell off immediately after.
Weir saw his side win just one of their next 12 fixtures in all competitions - a first-round Football League Trophy tie against Scunthorpe United on penalties - as they nosedived towards the bottom of the table.
United finished fifth in League One last term and although they were knocked out in the play-offs, they were expected to challenge for promotion this season.
Weir's side have done anything but though, losing seven of their 10 League One matches this season, form that sees them sit 22nd in the table.
And Sheffield United's board finally lost patience with Weir after a 1-0 home defeat to fourth-tier Hartlepool United in the second round of the Football League Trophy on Tuesday.
In a statement on the club's official website, chief executive Julian Winter said: "First and foremost all at Sheffield United, from the board right through the club, wish David and Lee (Carsley, Weir's assistant who has also left) all the best for the future.
"None of us wanted to be in this position at this point in the season."
Former United defender Chris Morgan - who managed the club on a caretaker basis last season - has again been appointed to the role.
Morgan's first task in charge of the club will come on Sunday when they face Coventry City.
