Leicester's record signing headed home in the 22nd minute to immediately cancel out Alexis Sanchez's first league goal for Arsenal since joining from Barcelona.
Ulloa's intervention proved enough to earn a valuable point as the visitors failed to find a winner in an entertaining encounter at the King Power Stadium.
Arsenal headed into the game desperate to cap a good week, having qualified for the UEFA Champions League group phase for the 17th successive season on Wednesday.
Yet an inspired Ulloa proved a handful and, with Leicester also defending doggedly, Arsene Wenger's men had to settle for a share of the spoils.
Arsenal have now collected a win and two draws from the opening three league fixtures of 2014-15, with Leicester taking two points from the same number of games.
Santi Cazorla's wonderful lobbed pass freed Yaya Sanogo and, after the striker's woeful effort was blocked by Kasper Schmeichel, the Chilean was left with a simple finish.
Yet Leicester found an immediate riposte through Ulloa, who met Schlupp's stunning cross with a powerful header that gave Szczesny no chance.
Arsenal continued to dominate possession after half-time, but were fortunate not fall behind when Ulloa chipped into the side-netting after brilliantly turning Chambers.
Leicester, though, were always on hand to fire warning shots of their own, David Nugent seeing a goal-bound effort blocked following good work from the outstanding Ulloa.
