The 22-year-old Spaniard was lambasted for failing to deal with a cross that led to an injury-time equaliser for Tottenham Hotspur in Sunday's 1-1 Premier League draw.
"It's been over the top. It is always over the top when it comes to criticism here," Ferguson, whose team face Fulham in the FA Cup on Saturday, was quoted as saying on the BBC. "You have to listen to some idiots in the game.
"I am not interested in discussing it because we know it is unfounded."
Ferguson was asked by the FA to explain comments he made in the wake of the match when he criticised assistant referee Simon Beck for not giving his side a penalty for a foul on striker Wayne Rooney.
"We are high profile and the profile of me is such that the FA naturally panic as soon as the press criticise them," said Ferguson, who will find out on Tuesday if he is going to be charged.
"Whether I think it is unfair or not doesn't matter to them really at this point.
"I just think it is more about me than what I have said."
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.