The striker was brought down by John O'Shea, who was subsequently dismissed, in the 78th minute and converted from the spot for his first Palace goal.
Steven Fletcher looked to have earned Sunderland a point with a second-half header that cancelled out Danny Gabbidon's early goal.
Substitute Stuart O'Keefe secured the victory with a third goal late on, taking advantage of the room in Sunderland's defence to fire home.
Jose Campana delivered a corner into a congested penalty area, Kagisho Dikgacoi challenged for the ball at the back post and Gabbidon got the faintest of flicks to send the ball past Keiren Westwood.
Ian Holloway's side dominated much of the opening period but failed to add to their lead going into the break, with Jason Puncheon and Mile Jedinak posing a real threat to Westwood's goal.
It took the 26-year-old 19 minutes to make an impact as he levelled with a well-taken header from Colback's cross.
The striker evaded the attentions of Jedinak to head home from an unmarked position inside the penalty area – his ninth goal away from home for Sunderland.
With 12 minutes to play, O'Shea gave Palace a lifeline as he brought down Gayle inside the penalty area, the Ireland international lunging for the ball but catching the striker.
O'Shea was sent off by referee Lee Probert and Gayle made no mistake from the spot, his penalty squirming under Westwood for his first goal in the top flight.
O'Keefe added a third in stoppage time, the 22-year-old curling his shot past Westwood from the edge of the penalty area to cap an important win for Palace.
