Villa, who had new signings Richard Dunne, James Collins and Stephen Warnock making their debuts in defence, shaded the first half with England midfielder James Milner spurning several chances.
Agbonlahor latched on to substitute striker John Carew's centre five minutes from time to head low past Birmingham goalkeeper Joe Hart for his second goal in two games.
Villa, who moved up to sixth with Birmingham in 15th, could have made it 2-0 but Agbonlahor shot over the bar with only Hart to beat.
Fulham produced a strong second-half performance to beat Everton 2-1 at Craven Cottage and climb to 10th.
Winger Damien Duff grabbed the winner 11 minutes from time with a rifled left-footed shot from outside the box, condemning Everton to a third league defeat of the season and leaving them second-bottom.
Tim Cahill had given Everton the lead on 33 minutes, heading in a Leighton Baines free kick before Paul Konchesky's deflected strike levelled the scores after the break.
Roy Hodgson's Fulham had scored just once all season before the match and survived a late goalmouth scramble to secure their second win of the campaign.
