Germany have maintained the momentum from their third place World Cup finish last year, winning all five of their qualifiers to top the group with 15 points, conceding only one goal and scoring 17 in the process.
Klose scored in his sixth consecutive international, tapping in from close range after three minutes and 2010 World Cup top scorer Muller struck twice before the break, both times set up by Real Madrid's Mesut Ozil.
"It was hugely important to carry that moment of the World Cup into this competition and we have done that," said Muller. "It was not always easy or beautiful to see for the fans with their defending but we did our job well."
Last-placed Kazakhstan, ranked 129 places below Germany, had to endure wave after wave of attacks but battled bravely in the second half to keep the score down until Klose netted his 61st goal in 107 matches two minutes from time.
Germany lead the group from Belgium while Austria can close the gap to five points with a win over Turkey on Tuesday.
"The first half was good and we had a handful of chances but in the second half we took our foot off the gas," said coach Joachim Low.
With nine players from Germany's World Cup team in the starting line-up and with the visitors missing six key players it was always going to be a one-sided affair.
Klose, back at his old hunting ground at Kaiserslautern where he played for five years, needed only three minutes to score when he was left unmarked at the far post to tap in a Bastian Schweinsteiger free-kick.
Ozil then turned on the style, curling a free kick for Mueller who scored with a glancing header on 25 minutes.
Muller was again on target just before the break when Ozil chipped a cross into the box, and he chested the ball down and volleyed home for his seventh goal in 14 internationals.
Klose added his second goal two minutes from time with a tap-in to move seven goals short of Gerd Muller's Germany record of 68 international goals, although the former Bayern Munich striker scored them in just 62 games.
