Allen led Gillingham to the League Two title last season, but they have struggled in the current campaign, failing to win any of their first 10 matches in all competitions.
Successive wins over Crewe Alexandra and MK Dons looked to have eased the pressure on Allen, but after a 2-0 defeat at Shrewsbury Town on Saturday, Gillingham's board acted to remove the 48-year-old.
In a statement released on the club's official website, Gillingham - who sit 17th in the League One table - said they felt a change in manager was "essential" in relation to the "long-term interests of the club."
Gillingham chairman Paul Scally said: "It is always a hugely difficult time when a managerial change occurs and this moment is no exception, and whilst the decision taken was made unanimously by the board, it was an exceptionally difficult one, but one which after long deliberation was felt absolutely necessary.
"I would like to wish Martin every success in his future career, I have no doubt he will be back managing very shortly and I have very much enjoyed the experience of working with him, on both a business and personal level over the past 16 months."
Allen was appointed at Gillingham on a two-year deal in July 2012.
