Miller, who had a season at Celtic Park in 2006, fired home a cracking volley to hand Rangers a 2-1 lead before rifling in his 13th league goal of the season from the penalty spot to secure the points midway through the second half.
"Kenny's been terrific for us and we didn't get close enough to him in the first half but we did in the second," Rangers manager Walter Smith told Sky Sports.
"Going into this match both teams had won eight of their matches and from that point of view the game was more important so I'm glad we won."
Celtic took the lead in a typically fiery encounter when an unmarked Gary Hooper stabbed home at the far post on the stroke of half-time.
Rangers, however, were handed a route back into the match when Kyle Lafferty's header was diverted into his own net off the chest of Celtic defender Glenn Loovens.
"Walter told us to be more confident on the ball and getting it moving quicker as we felt that we had started the better team," Miller said.
"I think we sat off them a bit and you saw in the second half when we got at them we scored three goals that gave us a bit of breathing space."
The result leaves Rangers three points clear of Celtic at the top of the table with 27 points from nine games, maintaining their perfect start to the campaign.
