Blatter has faced widespread criticism following his comments on Wednesday when he said in two separate TV interviews there was no racism in football and that players involved in any confrontations on the pitch should settle their differences with a handshake at the end of the match.
Taylor, speaking to Sky Sports News, said: "This goes beyond the line. To talk like he did shows he is totally out of tune, and out of time.
"He should move aside for [UEFA President] Michel Platini. If one person should get it about racism it is the head of FIFA which has 200 countries in the world, which are so diverse and have different backgrounds, colours, cultures and creeds and if he is not getting it then he's got to move on."
Asked if he should resign, he said: "I think without a shadow of a doubt."
Blatter's comments were televised on the same afternoon as the English FA charged Uruguayan striker Luis Suarez of Liverpool with racially abusing Manchester United defender Patrice Evra last month. Suarez is pleading not guilty to the charge.
The FA is also investigating allegations of racial abuse by England and Chelsea captain John Terry towards Queens Park Rangers defender Anton Ferdinand last month. Terry has denied the charge, but is also being investigated by the police.
There has been a predictably vehement backlash in England towards Blatter with The Sun's front page headline "Blind as a Blatt" making their feelings known and many pundits calling for him to quit.
European newspapers were less forthright, with some major ones like Italy's Gazzetta dello Sport not mentioning the controversy at all on their website, but Switzerland's Blick talked of a "racism storm".
ASTONISHED FERDINAND
Manchester United and England defender Rio Ferdinand - Anton's brother - was one of many critics, saying on Twitter that he was "astonished" by Blatter's comments.
"Sepp Blatter your comments on racism are so condescending its almost laughable. If fans shout racist chants but shake our hands is that ok?" Ferdinand wrote. "I feel stupid for thinking that football was taking a leading role against racism - it seems it was just on mute for a while."
Blatter gave interviews to broadcasters CNN and Al Jazeera.
Asked if there was racism on the pitch he told CNN World Sport: "I would deny it. There is no racism, there is maybe one of the players towards another, he has a word or a gesture which is not the correct one.
"But also the one who is affected by that, he should say that this is a game. We are in a game, and at the end of the game, we shake hands, and this can happen, because we have worked so hard against racism and discrimination."
He also said on Al Jazeera: "During a match you may say something to someone who's not looking exactly like you, but at end of match it's forgotten."
Blatter attempted to play down his remarks afterwards by issuing a statement where he pledged his commitment to stamping out ra
