The Welsh side looked to be in complete control in their first game following the sacking of boss Malky Mackay by controversial club owner Vincent Tan, after Jordon Mutch had opened the scoring with a deflected strike after just six minutes.
Three points appeared to be a formality when former Sunderland man Fraizer Campbell struck in the early stages of the second half.
There was still time for even more late drama and the comeback was completed with the last meaningful kick of the game when Colback found the net with a deflected effort to spark jubilant scenes in the away end and leave the Premier League's bottom side just two points adrift of safety.
The 22-year-old was instead booked for simulation, and Mutch was again involved in the game's first real chance shortly afterwards, the midfielder meeting Craig Noone's cross with a powerful header that flew over the crossbar.
Sunderland struggled to gain a foothold in the contest and owed much to their goalkeeper, Vito Mannone, who produced an excellent fingertip save to keep out Kim Bo-Kyung's fierce long-range drive.
However, it was Sunderland who perhaps should have scored the game's second goal just after the half-hour mark when Altidore somehow shot wide with the goal at his mercy after David Marshall had failed to deal with an effort from Ki Sung-Yueng.
Sunderland brought Fletcher on in place of Borini at the break - the Liverpool loanee apparently suffering an illness - and their task was made even harder 12 minutes into the second period when Campbell converted Mutch's pull back.
