The Dutch champions have amassed one point from their opening three group games, courtesy of a home draw with Milan at the start of last month.
Never before have Ajax failed to win in their first three fixtures of the competition, although a victory at Amsterdam ArenA against Celtic will lift them above the Scottish club in Group H.
Frank de Boer's side do not head into the match in particularly impressive form, their only win in five outings coming against fourth-tier ASWH in the KNVB Beker.
Ambrose said: "Going to Amsterdam is huge for us.
"It'll be a deciding game for us – every one we play from now on is crucial, a cup final, to make the last 16. We know what we're up against and we know we have to get everything right.
"It's important to believe we can do it and we're working hard to achieve our goals. We know we can't write Ajax off though. They played good football against us at home, but they didn't take their chances. We took ours.
"They're a good side but the most important thing is for us to keep our heads straight and stay focused because we know if we lose it's all over. We don't want to let that happen.
"We're comfortable going there now, even though it won't be easy. But they know they can't lose the game if they have any chance to qualify. The better team will win the game, but I believe we're the better team."
Neil Lennon's Celtic side have lost their last two European away matches, in the play-offs at Shakhter Karagandy and against Milan on matchday one, but the Scottish Premiership club did emerge 3-1 victors on their last visit to Amsterdam in 2001.
