Carroll was dismissed by referee Howard Webb in last Saturday's 2-0 victory over fellow strugglers Swansea City, after it was adjudged he swung an arm into the face of Chico Flores.
The Swansea defender went to ground and theatrically clutched his face, with replays suggesting Carroll's arm had brushed the top of his head.
West Ham launched an initial appeal, but an independent commission for the FA upheld the decision on Tuesday.
That led to outrage from the West Ham hierarchy, with manager Sam Allardyce saying his side had been the victim of "injustice" at his pre-match press conference ahead of Saturday's encounter with Aston Villa.
West Ham continued to dispute the decision and an FA arbitrary panel subsequently convened on Friday to reconsider the appeal.
But the original ruling was once more upheld, and Carroll is set to miss the trip to Villa, as well as crucial games with Norwich City and Southampton as West Ham battle relegation.
A statement from the FA read: "An Independent Arbitration Tribunal convened under FA Rule K has today dismissed a legal challenge brought by West Ham United FC and Andy Carroll in relation to the red card received by Carroll in the match between West Ham United FC and Swansea City FC on 1 February 2014.
"The Independent Tribunal resolved that there was no serious issue to be tried and also awarded The FA its costs."
Carroll - who has missed a huge chunk of the season due to a heel injury - had performed impressively in the Swansea fixture, setting up Kevin Nolan for both goals.
