The 58-year-old Northern Irishman announced his decision in a joint statement with chief executive Paul Faulkner on the club website.
GEAR:Get an Aston Villa shirt
"I have enjoyed my time at Aston Villa immensely. It's obviously a wrench to be leaving such a magnificent club," O'Neill said.
"I would like to pay tribute to the Villa players, my coaching staff and the Villa supporters for all the support and encouragement they have given both the club and me personally during my time as manager."
Reserve team manager Kevin MacDonald will take caretaker charge for the club's opening game of the season at home to West Ham United on Saturday.
Former Villa striker Gary Shaw criticised O'Neill's decision to quit.
"I don't know what has happened but the timing is not great, it's a bit naughty of Martin to leave the club in this situation just a few days before the start of the season," Shaw told Reuters.
"The club badly need two or three quality players bringing in as quickly as possible and I can't imagine Kevin MacDonald has licence to make these signings so they need to make an appointment very soon.
"I don't know who they should go for but it is a great job for someone. I have heard Glenn Hoddle and Martin Jol mentioned, maybe they wouldn't be bad options," added Shaw.
MEDIA SPECULATION
O'Neill gave no reason for his resignation but has had to deal with constant media speculation over the possible sale of midfielders James Milner and Ashley Young.
Last week O'Neill, speaking about the likelihood of Milner joining Manchester City, told the club website: "I think losing your best players... would obviously cause you a headache.
"That's a problem, it's a problem because you're trying to build something and something like this happens."
He had a similar battle over former captain Gareth Barry who was heavily linked in the media with a transfer to Liverpool in 2008 before joining Manchester City last year.
O'Neill's departure comes four years after joining the Midlands club. His tenure included three sixth-place league finishes and Villa's first final in 10 years - a 2-1 League Cup defeat by Manchester United last season.
"The club would like to thank Martin for the great work he has done at Aston Villa over the past four years," Faulkner said.
"He has helped to establish the club in the upper echelons of the Premier League, has taken us to Wembley and we have also qualified for European competition for the past three seasons under his management. We wish him the best for the future."
O'Neill, who took Leicester City to the Premier League in 1996 and also lifted two League Cups with them, went on to win three Scottish Premier League titles with Celtic as well
