The Welsh side looked to be in complete control in their first game following the sacking of boss Malky Mackay by controversial club owner Vincent Tan, after Jordon Mutch had opened the scoring with a deflected strike after just six minutes.
Three points appeared to be a formality when former Sunderland man Fraizer Campbell struck in the early stages of the second half.
But Sunderland refused to lie down, and they gave themselves hope when Steven Fletcher pulled a goal back with seven minutes left to set up a frantic finish.
There was still time for even more late drama and the comeback was completed with the last meaningful kick of the game when Colback found the net with a deflected effort to spark jubilant scenes in the away end and leave the Premier League's bottom side just two points adrift of safety.
There was worrying news for Sunderland, though, after reports that Fabio Borini had been taken to hospital after collapsing at half-time.
Gary Medel was restored to the Cardiff midfield as one of three changes made by interim coaches David Kerslake and Joe McBride, while Sunderland boss Poyet - without Adam Johnson through illness - selected Jozy Altidore to lead the line in place of Fletcher.
Cardiff made a bright start and they were denied a penalty in the first minute as referee Chris Foy ruled that Mutch had dived after a challenge with Valentin Roberge.
The 22-year-old was instead booked for simulation, and Mutch was again involved in the game's first real chance shortly afterwards, the midfielder meeting Craig Noone's cross with a powerful header that flew over the crossbar.
He was finally rewarded for his initiative after six minutes when he capitalised on ponderous defending from the visitors to burst forward and fire a shot that deflected off Modibo Diakite and into the bottom corner.
Sunderland struggled to gain a foothold in the contest and owed much to their goalkeeper, Vito Mannone, who produced an excellent fingertip save to keep out Kim Bo-Kyung's fierce long-range drive.
The chances continued to flow for the hosts as Steven Caulker, Peter Whittingham and Campbell all went close to adding to their lead.
However, it was Sunderland who perhaps should have scored the game's second goal just after the half-hour mark when Altidore somehow shot wide with the goal at his mercy after David Marshall had failed to deal with an effort from Ki Sung-Yueng.
Sunderland brought Fletcher on in place of Borini at the break - the Liverpool loanee apparently suffering an illness - and their task was made even harder 12 minutes into the second period when Campbell converted Mutch's pull back.
Lee Cattermole went close to finding a reply soon after but then Fletcher frayed the home nerves as he turned home an Emanuele Giaccherini cross seven minutes from time.
That set up a late Sunderland onslaught, with Fletcher and Roberge both going close, but it was Colback who proved to be the hero in the fifth minute of injury time as his effort from the edge of the box took a wicked deflection past a despairing Marshall.
