The Spaniard was dismissed just past the hour mark after his attempts to wrong-foot Sadio Mane ended with him losing control of the ball and then handling outside the box.
His replacement Jussi Jaaskelainen was given surprisingly little to do as Southampton's dominance of the ball produced few shots at goal.
With West Ham having conceded late in Sunday's 1-1 draw against Manchester United, and Southampton having scored in stoppage time to beat QPR last weekend, there was obvious tension in the visiting ranks, but Sam Allardyce's 10 men stood firm.
Allardyce was given cause for concern, though, as Andy Carroll spent the final few minutes limping around the pitch after picking up a knock.
Southampton started brightly and James Tomkins, who signed a new long-term deal on Tuesday, underlined his value to the side with a well-timed intervention to deny Mane as the forward looked to latch on to Graziano Pelle's flick following some impressive build-up play.
Mane's direct running caused problems again in the 13th minute when he raced through on goal and looked to find the net from a narrow angle, only for Carl Jenkinson to clear after Adrian had been taken out of the equation.
It was not all one-way traffic, though, and West Ham sounded a warning shot at the other end when Morgan Amalfitano's goal-bound effort was hacked away by Steven Davis.
But it was Southampton who looked the most likely and Mane dragged a shot wide from the edge of the box eight minutes before the break to highlight that fact.
Carlton Cole, making his first start since November, was perhaps fortunate to escape a red card when he caught Maya Yoshida in the face with a high boot.
The defender received a nasty cut just below his eye, but continued after treatment, while Cole's evening was ended not long after in shambolic circumstances.
Adrian attempted to evade the onrushing Mane when a simple clearance would have averted the danger and, with the Senegal international having got a touch, the goalkeeper sought to retrieve the situation by diving on the ball outside the area, earning him a red card.
Cole made way as Jaaskelainen came on, but the Finn was surprisingly kept quiet initially, watching on as Pelle fired well over the crossbar when well placed.
Southampton continued to force the issue, though, and substitute Dusan Tadic looked to have an opportunity to shoot when he collected James Ward-Prowse's deep cross, but the Serbian dallied and the chance was gone.
Pelle's acrobatic volley looped up in the air and into the grateful arms of Jaaskelainen as the hosts struggled to test the veteran.
Filip Djuricic, a loan signing from Benfica, was brought on for his Southampton debut for the final 10 minutes, though his first 30 seconds on the field were spent watching a pitch invader, who was eventually escorted away by stewards.
Tadic showed good feet in the area, but could not get a shot away and Jose Fonte finally forced a good save from Jaaskelainen in the first of five minutes of time added on, but there was no way through for Southampton, who slipped to fourth.
‘Maybe I’ll be signed by Wrexham, who knows? English football would suit me – if an offer came, I wouldn’t think twice, I’d go immediately’ Euro 2024 cult hero reveals his UK dream
‘Managing Leeds? It was an option that appeared, but it wasn’t the right timing. I decided it wasn’t a good idea to leave the club I was at mid-season’: Premier League boss admits to turning down opportunity to replace Jesse Marsch in 2023
