Pelle produced a composed close-range finish in the 88th minute of an entertaining encounter at the Britannia Stadium as Ronald Koeman's men, who are second in the Premier League after nine games, reached the quarter-finals for the first time in 10 years.
The sides headed into the game with plenty of familiarity after Stoke fell to a 1-0 loss at St. Mary's Stadium in the Premier League on Saturday.
A return to their home venue did not make much difference to Mark Hughes' men early on as Pelle grabbed his eighth goal of the campaign with a superb strike.
Southampton appeared to be cruising when Shane Long added another later in the opening half, only for Steven N'Zonzi to pull one back just after the break.
The pendulum swung again when Mame Biram Diouf levelled in the 83rd minute, yet Pelle ended any hopes of a home victory with his second of the night after Stoke substitute Peter Crouch had been sent off for a second bookable offence.
Southampton began like a side brimming with confidence and took the lead in just the sixth minute.
Pelle took full advantage of being afforded too much space when superbly curling a 25-yard effort off the inside of Asmir Begovic's left-hand post and into the net.
Stoke - boasting an attacking line-up with Bojan Krkic, Marko Arnautovic and Diouf brought into Hughes' XI - were unable to carve out a clear-cut opportunity as they sought to respond.
The hosts then fell further behind in the 31st minute when Long - one of the new faces in a Southampton line-up featuring four changes from the weekend - turned Steven Davis' cross through the legs of Begovic to cap a superb passing move.
Jonathan Walters had a goal ruled out for offside, before Pelle almost made things worse for Stoke, uncharacteristically blazing a half-volley over the bar after being found by Dusan Tadic's clever ball.
Whatever Hughes said to his players at half-time appeared to do the trick as N'Zonzi, capitalising on the visitors' defenders backing off, unleashed a powerful strike into the bottom-left corner from just outside the box.
Yet the home side were then left thanking their goalkeeper as Begovic twice rushed from his line to deny Long and Victor Wanyama.
Stoke spurned a great chance midway through the second half as a two-on-one went to waste when Diouf was somehow not picked out by the disappointing Bojan, who was unsurprisingly withdrawn soon after.
The home side continued to put their opponents' goal under siege and, having cleared a series of corners with last-gasp challenges, Southampton finally succumbed when Diouf headed in at the back post after Phil Bardsley had flicked on Charlie Adam's set-piece.
There was still to be another twist in the tale as Crouch, who had replaced Bojan, picked up a second yellow card in quick succession.
It was to prove a defining moment, too, as the ball dropped to Pelle off Erik Pieters from the resulting free-kick, and the striker instinctively placed the ball beyond Begovic.
