The defending champions beat St Johnstone 2-1 as Inverness suffered their first defeat of the campaign, going down 1-0 at Aberdeen.
Finnish forward Teemu Pukki scored on his home debut to give Celtic an 11th-minute lead at Celtic Park, the second league match in a row in which he has found the net.
Charlie Mulgrew effectively secured the three points 15 minutes later as he fired a low shot past goalkeeper Alan Mannus.
Celtic wasted a host of chances and their visitors pulled a goal back in the final stages. Liam Caddis rose the highest in the area to meet a cross and he headed the ball past Fraser Forster, but the home side survived a nervous ending.
Neil Lennon's men now have 16 points but do have the advantage of having a game in hand on Inverness, who remain top courtesy of goal difference.
The league leaders succumbed to Scott Vernon's 81st-minute strike as Aberdeen moved up to third, three points behind the early pace-setters.
Hibernian moved into the top half of the table thanks to their routine 2-0 win at home to struggling St Mirren. The visitors went into the match having picked up only one point from their first five games and goals from Jamie Collins and Paul Heffernan saw them lose once again.
Partick Thistle dropped down a place into sixth due to a combination of Hibernian's win and their own 1-1 draw with Kilmarnock.
They took the lead early on as Aaron Muirhead converted a penalty only for Kris Boyd hit back in the second half to earn a point for the visitors, who are still looking for their first win.
Hearts conceded two late goals to lose 2-1 at Ross County and remain a long way from safety at the foot of the table following their 15-point deduction at the beginning of the season.
Callum Paterson's 24th-minute goal looked set to give them victory but Melvin de Leeuw and Richard Brittain won the match for the home side in the last two minutes.
