Emanuele Giaccherini's first home goal since joining from Juventus and substitute Valentin Roberge's second-half header sent Sunderland into the fourth round as they put Sunday's dismissal of Paolo Di Canio firmly behind them.
The Italian's brief and controversial reign came to an end after a winless start to the campaign that left Sunderland at the foot of the Premier League table.
The visitors were without suspended duo Nathaniel Knight-Percival and Mark Little following their red cards in Saturday's 2-1 victory over Milton Keynes Dons, who Sunderland beat in the last round.
Peterborough, who scored 11 goals in the opening two rounds, suffered a further setback inside the opening 10 minutes as forward Lee Tomlin, the competition's top scorer with five strikes, hobbled off following a strong challenge from Ki Sung-Yueng, which earned the Korean a yellow card.
It was the home side who continued to make most of the early running and Giaccherini should have done better after latching onto Jack Colback's searching cross, but the Italian looped his effort over the bar.
Sunderland got their deserved breakthrough after 32 minutes though, when Giaccherini fired home Cattermole's cross on the half-volley.
The game should have been put out of the League One side's reach early in the second half, with Giaccherini and Altidore both failing to capitalise on presentable opportunities.
Peterborough sounded a warning at the other end as Britt Assombalonga, who has eight goals this season, had the ball taken off his toes by Sunderland goalkeeper Kieren Westwood.
