Gabriel Agbonlahor's classy goal on the counter attack set Villa on their way early in the first half but Jay Rodriguez headed Southampton level just after half-time.
Lambert's men showed their mettle, holding out against a second-half barrage from Southampton and hit the front again through Libor Kozak's close range header just after the hour mark.
And although Dani Osvaldo's header in the 69th minute made it all square at St Mary's Stadium on Wednesday, Delph's thunderous first league goal of the season, 11 minutes later, settled matters and ended Southampton's unbeaten home record this season.
"We were outstanding throughout the whole game," Lambert told Villa's official website.
"The first goal was first-class. Gabby showed great composure and it was a great finish.
"And what a strike from Fabian Delph! The strike was worthy of winning any game."
