Sherwood had admitted in the build-up to the game that uncertainty around his future made him feel like a "supply teacher" amid claims from midfielder Sandro that the coach would be disposed of in the close-season.
The 45-year-old also revealed that some of his players have been informed by their agents that a new coach will be at the helm next season.
Emmanuel Adebayor grabbed the third from the penalty spot towards the end of the first period and although Villa were slightly improved after the break, Paul Lambert's men rarely showed anything to worry the hosts.
One of most positive aspects of Sherwood's reign has been the resurgence of Adebayor and the forward set up Harry Kane twice in the opening 10 minutes, but both times Brad Guzan denied the England Under-21 international.
The American's efforts went unrewarded in the 14th minute, however as Gylfi Sigurdsson's pass was flicked into the path of Paulinho by Kane and the Brazilian slotted home at the second attempt.
Despite falling behind, Villa showed no appetite to draw level and found themselves 2-0 down in the 35th minute as Baker knocked Danny Rose's drilled cross past his own goalkeeper.
Three minutes later, Gabriel Agbonlahor made his side's position even worse as he blatantly blocked a shot inside the area with his elbow and Adebayor smashed in the resulting penalty.
Sherwood's reportedly fractious relationship with Sandro has lingered throughout his time in charge, but neither showed any sign of ill-feeling toward the other as the Brazilian was replaced just after the hour mark.
