Adebayor earlier this month announced he would quit the national team, saying he was haunted by the attack on the squad bus at the African Nations Cup in Angola in January, in which an assistant coach and press officer were killed.
Togo, shattered by the attack, withdrew from the competition and the Confederation of African Football (CAF) banned them from the next two tournaments as well as fining them $50,000, causing outrage in that country and elsewhere.
Adebayor, who returned to play for Manchester City after the attack, described the punishment as "monstrous" and called for CAF President Issa Hayatou to resign.
"Everything has been said already, everything has been done," he said.
Asked about Adebayor, Hayatou said: "If he refuses to play in the national team, it is a loss but it is his own decision. It is unfortunate...it is a loss for Africa but one cannot oblige him to do what he does not want."
"The vote is secret. You don't know who I voted for and I don't think you need to speculate," he told a Ghanaian journalist. "You are not as old as my son, you cannot shoot at me like this."
"This argument that while there is misery, well let all of the people be miserable cannot be the right argument. We are entitled to orchestras, to sporting events, to church gatherings, to economic forums - and not just this country, but the whole continent," Jordaan said.
"Football is a giver of hope and life and we must never argue that we must deny Africans the fundamental pleasure and joy that football generates," he said, adding that football was "the one expression where Africa can compete equally with anyone in the world."
