The visitors equalled a 30-year Bundesliga record of 36 matches unbeaten but had to do it the hard way after falling behind in the 34th minute, when Manuel Neuer's mistake allowed Niklas Sule to put the home side in front.
Bayern responded five minutes later as Franck Ribery's free-kick took a deflection off Mario Mandzukic and nestled in the bottom corner.
The win also meant Guardiola's side surpassed a 45-year-old club record for the longest undefeated start to a Bundesliga season, extending their run this season to 11 games to break the record set in the 1968-69 season, when the club claimed their first Bundesliga title.
"We had a bit of luck," he said.
"It certainly was not our best performance but we won. Not everything went perfectly, so we will need to correct the way we play."
"The 1-0 goal was my fault," he said.
"I do not know how that could happen, it was a gift for Hoffenheim.
"We came back, made it 1-1 then in the second half we took the lead. Since we had many opportunities, we deserved to win."
