Phil Bardsley was Sunderland's matchwinner with his first Premier League goal of the season in the 21st minute.
The defender perfectly placed a shot past Costel Pantilimon in the Manchester City goal, but Sunderland had to hang on to claim the victory.
Sergio Aguero saw a first-half header fly wide of goal, while substitute Jesus Navas had a chance blocked by Bardsley on the line.
City pushed hard for an equaliser, penning the hosts inside their own half, but Micah Richards and Eden Dzeko were both denied by Vito Mannone.
The three points lift Sunderland to within two points of Stoke City in 18th position, while Manuel Pellegrini’s side lose further ground on the leaders.
Pellegrini made five changes from the UEFA Champions League win over CSKA Moscow, with three of the alterations coming in his back four.
Wes Brown started his first Premier League game in 22 months for Sunderland, while Steven Fletcher and Sebastian Larsson both returned.
Despite an equal share of possession it was the hosts who had better of the early chances, but in the 18th minute they were lucky to see Larsson stay on the pitch after a mistimed challenge on Javi Garcia.
The tackle did not bring a card, but the Spaniard was withdrawn at half-time.
Three minutes later Poyet's side broke the deadlock through Bardsley.
The defender shrugged off the attentions of James Milner and Demichelis, who got in each other’s way on City’s right, and placed a composed finish into the far corner.
Just after the half hour City had their first sight of goal through Sergio Aguero, but the Argentina international wasted a fantastic chance.
After being released by Samir Nasri on the left, Aleksandar Kolarov delivered a cross to the back post for the 25-year-old, yet he was unable to get his back-post header on target.
The visitors were denied an equaliser five minutes into the second half as Bardsley blocked a shot from Navas, who replaced Garcia, on the goalline.
Aguero rolled away from John O'Shea on the left and raced into the penalty area before rolling the ball back to Navas, but his shot was blocked by the chest of the defender.
City's dominance intensified in the closing stages but Richards saw a low shot from the edge of the penalty area stopped by Mannone, and substitute Dzeko's first effort was parried away by the Italian prior to a frantic finale that saw Sunderland hold on.
'I think that’s the respectful thing for two captains to do after a game. I probably boiled over a bit at the end, with a bit of everything coming out': Jarrod Bowen lifts lid on heated clash with Mario Lemina after West Ham's win over Wolves
Arsenal want Dan Ashworth - for a VERY different kind of revolution
