Holloway, who took the job in January 2014, leaves the London club with Millwall second bottom of England's second-tier, eight points adrift of safety.
Under-21 coach and all-time leading goalscorer Neil Harris will take over until the end of the season, with his first game a visit to Bolton Wanderers on Saturday.
Millwall chairman John Berylson said: "This was a hard decision to take because we very much hoped that Ollie would prove to be the man to take us forward and on to the next level.
"I would like to thank him for his efforts, and in particular for keeping us in the Championship last May.
"This season, though, has proved to be an extremely challenging one and we now find ourselves facing another uphill battle to avoid relegation.
"Ian Holloway remains a fine manager and it was his impressive track record which led us to appoint him some 14 months ago. However, our view is that the decision to part company at this stage is in the best interests of the football club going forward.
"Neil Harris has agreed to take over managerial duties for the remainder of the season and I am confident that Millwall fans will give him every support over the next couple of months as we strive to turn our current run of form around."
