Arnautovic, restored to Stoke's starting XI but without a league goal since May last year, had the final word at Upton Park having twice seen efforts ruled out for offside.
A seventh-minute free-kick from Aaron Cresswell looked to have earned West Ham back-to-back home wins for the first time since December.
But substitute Mame Biram Diouf struck the post in an impressive late rally from the visitors before Arnautovic ensured further misery for home manager Sam Allardyce, who this week cast doubt on his long-term future at Upton Park.
Kouyate sauntered past a pitiful challenge from Steven N'Zonzi to test the Bosnia-Herzegovina goalkeeper in the 16th minute, although Winston Reid was forced into a last-ditch block to deny Jonathan Walters as Stoke belatedly stirred.
Mark Hughes' team were carrying the greater attacking threat by the time Walters cut in from the left to ripple the side netting and home goalkeeper Adrian turned a Moses snapshot behind six minutes before the break.
Charlie Adam dragged wide from the edge of the area under pressure from James Collins before the midfielder - who scored a stunning goal from the halfway line in last weekend's loss to Chelsea - picked out Arnautovic to slot home a tidy finish from an offside position.
West Ham's sharpness in attack diminished in Sakho's absence, but Kouyate found space in the box after 71 minutes and forced Begovic to beat a low strike out of the bottom corner.
At the other end, Adrian found himself comfortably beaten when he gratefully saw Diouf's well-directed effort cannon back off the upright.
Arnautovic cut a furious figure when the assistant's flag curtailed his celebrations for converting substitute Stephen Ireland's pass, but his joy was unchecked in the fifth and final minute of added time.
