Sherwood was appointed as Villa boss last month, but has failed to immediately improve the fortunes of the strugglers, with defeats to Stoke City and Newcastle United in his brief tenure so far.
The former Tottenham boss was reportedly set to take over at The Hawthorns in the close-season until talks broke down, and he was in the running once again when Alan Irvine was sacked in December, only for Tony Pulis to be given the job.
"Is there any extra needle in this match for me? Not at all," Sherwood said, ahead of Tuesday's meeting between the two sides at Villa Park.
"Good little club isn't it? It just wasn't right for me and wasn't right for them. I am delighted to be at this massive club."
Villa lie 19th in the table, on a run of seven successive defeats and without a win in 12, since a 2-1 success over Leicester City on December 7.
"I think we are very unfortunate to have no points from the two games but that is where we find ourselves," Sherwood said.
"All we are worried about is winning that first game.
"If we do that it will take us out of the relegation zone and then we will never look back.
"Everyone believes there is still enough time. It is about getting that win as quickly as possible.
"We are not feeling sorry for ourselves. This game against West Brom is everything to us."
