The visitors equalled a 30-year Bundesliga record of 36 matches unbeaten but had to do it the hard way after falling behind in the 34th minute, when Manuel Neuer's mistake allowed Niklas Sule to put the home side in front.
Bayern responded five minutes later as Franck Ribery's free-kick took a deflection off Mario Mandzukic and nestled in the bottom corner.
Thomas Muller netted the winner 15 minutes from time as Bayern returned to the top of the table.
The win also meant Guardiola's side surpassed a 45-year-old club record for the longest undefeated start to a Bundesliga season, extending their run this season to 11 games to break the record set in the 1968-69 season, when the club claimed their first Bundesliga title.
Despite that achievement, Guardiola conceded his side had been fortunate to claim all three points.
"We had a bit of luck," he said.
"It certainly was not our best performance but we won. Not everything went perfectly, so we will need to correct the way we play."
Meanwhile, goalkeeper Neuer was contrite in admitting fault for the Hoffenheim goal.
"The 1-0 goal was my fault," he said.
"I do not know how that could happen, it was a gift for Hoffenheim.
"We came back, made it 1-1 then in the second half we took the lead. Since we had many opportunities, we deserved to win."
