A 1-1 draw after extra time forced the match to penalties and the pressure proved too much for the hosts who missed three of their four spot kicks while Mali scored their first three.
Mali's shootout success sealed a strong comeback from last year's bronze medal winners after South Africa had dominated the early stages and taken a 31st-minute lead.
Tokelo Rantie scored with a tap in at the back post after a cross from Thuso Phala in a bright start by Bafana Bafana, who played at a high tempo and won repeated midfield battles to the delight of the near 60,000 crowd at the Moses Mabhida Stadium.
But when Seydou Keita equalised 13 minutes into the second half, the hosts lost control and allowed the game to slip away.
Mali caught South Africa cold after a quick midfield burst by Samba Sow allowed Mamadou Samassa to cross for Keita to net with a free close-range header as the home defence backed off.
PATIENT MALI
The goal seemed to suck the energy out of South Africa, silenced the crowd and allowed Mali to take control and dictate the pace for most of the rest of the game.
"The South Africans posed a lot of problems in the first half and we had to change the tactics in the second half to get back into the match," said Mali coach Patrice Carteron.
"They pressed us with a lot of diversity. But we knew we had to be patient and we had time to fight back."
The midfield battle continued but chances were limited and the game lost its flow as both sets of players tired quickly in the high humidity.
The two sides were restricted to long-range shots with May Mahlangu forcing two good saves from Mali goalkeeper Soumbeyla Diakite and Cheikh Tidiane Diabate coming on to trouble the South Africa defence.
Diakite, Mali's hero when they beat co-hosts Gabon at the last Nations Cup on penalties a year ago, made two saves in the shootout to keep up his impressive record in the tournament.
South Africa's exit continues the trend of host nations failing for a fourth successive edition.
