Samir Nasri fired City ahead from James Milner's pass midway through the first half but the Dutch side levelled on the stroke of halftime when Siem de Jong crisply dispatched Ricardo van Rhijn's cross from the right.
Defender Niklas Moisander guided in a corner to restore Ajax's lead on 57 minutes and there was no way back for City when Christian Eriksen's shot took a deflection off Gael Clichy to wrongfoot Joe Hart 11 minutes later.
Borussia Dortmund top Group G on seven points from three games after beating Real Madrid 2-1. Real have six points followed by Ajax (three) and City (one).
City manager Roberto Mancini, whose side suffered a group stage elimination in their debut campaign in the continent's elite competition last season, said it needed a "miracle" for the Abu Dhabi-owned club to now reach the last 16.
"They were better than us. They played better football. It is my fault, I didn't prepare well for the game and I take the blame," the Italian said.
"I thought in one way and it was different. When you prepare in one way and it is a different game it is difficult.
"We had more chances to score and in the Champions League you need to score. It is my fault, I repeat. We change for five minutes to three at the back, but we always have 11 players.
"I don't think that is important. Three, four, five, six or seven defenders. If someone wants that as an excuse than OK, but it's not the reason. It is very difficult to qualify now. It would be a miracle."
Defender Micah Richards appeared to question Mancini's tactics, telling Sky Sports: "Three at the back is something that we have not worked on a lot. It is a hard system because we are not used to it, I think the players prefer a 4-4-2 but he is the manager and we'll do what he says."
Ajax manager Frank de Boer said his side's equaliser late in the first half had come at a crucial moment.
"That changed a lot for us," said De Boer. "Now we could underline the things that went wrong in the first half and go for a win in the second half.
"We wanted to see a team that dared to play our football, and that is what I saw. Then you can lose a match, but if you play like that the chance you will win is bigger."
Victory was Ajax's first at home against an English opponent in 32 years, the last coming against Nottingham Forest in 1980.
‘After Manchester City’s recent form, maybe they’re the underdogs against Manchester United!’ Former Red Devils defender on this weekend’s derby
‘Arteta, Alonso, Emery, me… none of us were physical players – we needed the understanding of the game. That probably helped us move into management’: Premier League boss reveals reasons for natural career progression
