Having fallen to Group H defeats against Milan and Barcelona, triumph over the Eredivisie outfit represents a huge boost to the Scottish champions' European hopes.
But it was tough going at times for Neil Lennon's men, with Christian Poulsen striking the post for the visitors before Forrest converted from the spot thanks to Stefano Denswil's 43rd-minute foul on Anthony Stokes.
Beram's speculative deflected drive early in the second period sent the home crowd into raptures, while Thulani Serero spurned Ajax's best two chances and drew a rash tackle from substitute Nir Biton that meant Celtic finished with 10 men before Lasse Schone netted an emphatic consolation as the game's final act.
Debate over Forrest's spotting of the ball ensued, with Veltman, Poulsen and Celtic's Virgil van Dijk booked for their part in the argument, but the 22-year-old kept his cool to drill the kick beyond Jasper Cillessen's dive into the top right corner.
Fraser Forster produced a stunning save to thwart Serero following the Ajax midfielder's slick one-two with Kolbeinn Sigthorsson, before Celtic claimed an unlikely two-goal cushion in the 54th minute.
Siem De Jong fizzed a drive from distance narrowly past the far post as Ajax sought a route back into the game, while Kayal wonderfully spun Serero before thudding another well-hit effort into Cillessen's chest as the game entered its final quarter.
Serero found space in the Celtic box with 11 minutes remaining but missed his kick before blazing over and Ajax reached the halfway point of the group stage with a solitary point despite Biton's late censure and fellow sub Schone's last-gasp intervention.
