Andreas Weimann had scored Villa's first goal in 547 minutes to put them in front in the first half and they looked set to end a run of five consecutive defeats by securing an elusive victory.
The game changed 25 minutes from time, though, when Benteke inexplicably raised his hand into Ryan Mason's face off the ball and was shown a straight red card.
An unmarked Nacer Chadli converted an Erik Lamela corner from close range six minutes from time to draw Mauricio Pochettino's side level at Villa Park.
And Harry Kane's deflected free-kick in the 90th minute rubbed salt into the Villa wounds as Tottenham secured only their second win in eight top-flight games.
It was Kane's ninth goal in 13 appearances across all competitions this season and will lead to more calls for the England Under-21 striker to be given his first Premier League start under Pochettino.
Charles N'Zogbia and Nathan Baker returned to Villa's starting line-up at the expense of Gabriel Agbonlahor and Ciaran Clark.
There were also two changes to the Tottenham side beaten by Newcastle United last weekend, as Roberto Soldado and Kyle Naughton replaced Lamela and Eric Dier, who was ruled out due to illness.
Benteke ought to have put Villa in front after 10 minutes, but the Belgium striker stooped to head an inviting cross from Matt Lowton wide.
Both sides looked dangerous going forward and Soldado somehow failed to hit the target with a header from close range after Christian Eriksen's shot had deflected into his path.
The lively N'Zogbia then cut the ball back for an unmarked Benteke, who took a touch before unleashing a venomous left-foot strike that rattled the post.
Having ridden their luck, Tottenham were unable to prevent Weimann from opening the scoring 16 minutes in.
N'Zogbia was the architect, as he surged into the penalty area from the right touchline unchallenged and picked out the onrushing Austrian forward, who slid over the slick turf to finish with his right foot from six yards out.
Younes Kaboul spurned a great chance to equalise six minutes later, failing to get a proper contact on the ball after it had dropped nicely for the defender inside the six-yard box.
Villa looked vulnerable at the back and they were indebted to goalkeeper Brad Guzan for making a fine diving save to deny Emmanuel Adebayor after the Togo striker had raced clear.
Tottenham were struggling to contain Benteke, who again came close to opening his account for the season when he rose above Kaboul but headed just wide three minutes before half-time.
Pochettino replaced Eriksen with Lamela at the break and Tottenham came close again six minutes into the second half, when Adebayor was unable to bundle the ball home after Guzan had spilled a Chadli cross.
Kane also came on for Adebayor 12 minutes after the interval and made an immediate impact by forcing a save out of Guzan from a tight angle, then heading straight at the United States keeper from the resulting corner.
Jan Vertonghen was shown a yellow card for a challenge on Ashley Westwood when Villa fans wanted a stronger sanction.
Referee Neil Swarbrick did brandish a red 65 minutes in, though, as Benteke was punished for raising his hand into Mason's face as tempers frayed.
Chadli escaped with a yellow card for a poor tackle on Westwood and the Belgium winger then stunned Villa when he prodded in at the back post after he was left unmarked from a Lamela corner.
And Kane sealed a dramatic late fightback when his free-kick deflected off Baker and gave the wrong-footed Guzan no chance.
