Right-back Clyne fired home from 30 yards out five minutes before half-time to ensure that Ronald Koeman's impressive start to his Southampton tenure continued with a third-round victory over a much-changed Arsenal side.
Alexis Sanchez had put Arsenal in front with a stunning free-kick, but Arsene Wenger's wait to win this competition for the first time goes on after the visitors mounted a fightback for a win which was no more than they deserved.
Dusan Tadic equalised with his first goal for the club from the penalty spot after Tomas Rosicky had brought down the impressive Sadio Mane and Clyne's thunderbolt put the south-coast club through.
Wenger made 10 changes to the side that beat Aston Villa in the Premier League on Saturday, with Abou Diaby making his first Arsenal start in 19 months after recovering from a knee injury, while goalkeeper David Ospina was handed a debut following his close-season move from Nice and Calum Chambers faced his former club.
Southampton looked dangerous going forward and it came as no surprise when they pegged Arsenal bag after 20 minutes, when Tadic calmly slotted home a penalty after Rosicky had upended the lively Mane.
The equaliser was fully justified for the visitors and they were in front five minutes before half-time, as Clyne let fly with a venomous right-footed strike from 30 yards out which Ospina failed to keep out after a Tadic corner had been cleared into the defender's path.
Forster then got down well to keep out a Lukas Podolski strike at his near post to ensure Koeman's side led at the break.
Tadic was a constant menace to the home side and Ospina had to be alert to palm away the Serbian's left-footed free-kick midway through the second half.
Graziano Pelle ought to have sealed it four minutes from time when he headed Davis' corner over the crossbar, but Arsenal were unable to make the striker pay for that miss.
