Caceres maintains perfect Juve start
The Uruguay right back, on loan from Barcelona, smashed home a loose ball on the edge of the box midway through the second half.
David Trezeguet, starting up front in place of the rested Vincenzo Iaquinta and the injured Alessandro Del Piero, secured a battling victory late on when he tucked in from close range.
Juve's third win from three came at a cost, though, with new playmaker Diego, who scored twice in the 3-1 win over AS Roma on their previous visit to the capital last month, going off before the break with a leg injury.
Lazio felt hard done by on the stroke of halftime when Stefano Mauri's scrambled goal was ruled out for a foul but Juve could have won by more had Amauri managed to connect with debutant Fabio Grosso's cross.
AC Milan were earlier held to a dull 0-0 draw at promoted Livorno.
Leonardo's side, hammered 4-0 by Inter Milan in the derby before the international break, gave a first start to former Real Madrid striker Klaas Jan Huntelaar with Marco Borriello injured but they could not break down an organised defence.
Milan, who won their opening encounter at Siena, lacked creativity early on with Andrea Pirlo spending the first half on the bench after recent exertions for Italy.
He came on for the again ineffective Ronaldinho early in the second period and soon hit the bar with a free kick but it was the closest the visitors came. Huntelaar was also replaced by Filippo Inzaghi.
Livorno, who had two players with their names spelt incorrectly on the back of their shirt, threatened sporadically and could easily have won.
Stuttering Milan must now prepare for their return to the Champions League in the clash at Marseille on Tuesday, when Juve host fellow French side Bordeaux.
Former Juve coach Claudio Ranieri takes charge of Roma for the first time at Siena on Sunday while champions Inter host Parma.