"I believe England didn't understand the ground rules when we went in - and paid the price," he told a conference in Qatar, referring to the failed attempt to win the right to stage the 2018 World Cup which will now be hosted by Russia.
"Why couldn't they [FIFA] have said, we want to take it to the Gulf, to the eastern bloc? We spent 19 million pounds on that bid," added Richards in a reference to Qatar also winning the bid for the 2022 finals.
"When we went for it everybody believed we had a chance. But as we went through it a pattern emerged that suggested maybe we didn't."
Richards, who reports said had later injured himself falling into a fountain, was also quoted by media as saying FIFA "stole" football from the British.
"England gave the world football. It gave the best legacy anyone could give. We gave them the game," Richards said.
"Then, 50 years later, some guy came along and said, you're liars, and they actually stole it. It was called FIFA.
"Fifty years later, another gang came along called UEFA and stole a bit more."
The English FA issued a statement distancing itself from Richards, whose comments were made in front of FIFA members.
"Sir Dave Richards is not representing The FA at this conference and his personal views are in no way shared or endorsed by The FA," it read.
"The FA greatly values its relationships with FIFA and UEFA, which it is working hard to strengthen."
The Premier League also said Richards was speaking in a personal capacity.
ALCOHOL ISSUE
Richards said players and fans would find it difficult to cope with the extreme summer temperatures of the Gulf in 2022.
"I think Qatar did a fantastic job in getting the World Cup," he added. "The big downside is the weather. Can we play a World Cup in June?
"There are thousands and thousands of fans to consider and you have to look at it. My medical people in England have talked about the effect [the heat] will have on the players but you've also got to look beyond the players to the fans."
Richards said the availability of alcohol was another issue that would need to be addressed in advance of 2022.
"If you don't do something about it you're starting to bury your head in the sand a bit," he explained. "You've got to address it.
"In terms of [balancing] cultures there's got to be a happy medium. The English will acknowledge the culture of the Gulf but in England [a beer] is our tradition, it's part of our heritage."
Earlier on Wednesday, 2022 Supreme Committee Secretary General Hassan Al Thawadi said Qatar was considering whether to allow alcohol in stadiums at the World Cup before adding he did not see a need for it.
