Nan Yong will be replaced by Wei Di, the former director of water sports at the sports ministry, as director of the CFA, deputy sports minister Cui Dalin told a news conference reported by the official Xinhua news agency and in local media.
"We firmly support the police's crackdown on the criminal activities of illegally manipulating games and soccer gambling," Cui, tipped to be the next chairman of the CFA, said.
"Anyone involved in soccer that commits these crimes will be seriously punished," he added.
Nan was taken away by police with vice president Yang Yimin, who has also lost his job, and the CFA's head of women's football Zhang Jianqiang.
They were asked to "clarify some facts in several important cases of soccer gambling and illegal manipulating domestic soccer league games by using business bribery."
A former speed skater, Nan was best known for organising the successful women's World Cup finals in China in 2007 and replaced the much-derided Xie Yalong last year.
More than 20 officials, players and club managers, including Xu Hongtao, the president of Chengdu Blades, owned by England's Sheffield United, have been arrested or detained in the past two months on suspicion of matchfixing or gambling, which is illegal in China.
The probe into matchfixing followed a string of comments in recent months by top Communist Party officials on the need to clean up the men's professional game in China, which is widely perceived as being riddled with corruption.
