Delph opened the scoring in the 51st minute with a well-taken strike and on-loan Manchester City winger Scott Sinclair sealed Villa's progress to the semi-finals before the closing moments of the contest descended into farce.
Both sides finished with 10 men as West Brom defender Claudio Yacob and Villa defender Jack Grealish both received controversial second yellow cards.
Yacob departed prior to Sinclair's goal and following Grealish's red card, a group of Villa fans entered the field of play, with one momentarily stealing the corner flag.
Supporters streamed onto the pitch from all sides of Villa Park at the final whistle and Delph found himself caught up in the rush.
"Very, very scary," the England midfielder told BBC Sport when asked for his assessment, although he understood the outpouring of emotion from the Villa faithful amid a largely forgettable season to date.
"My armband got nicked, someone got my left boot, but I could appreciate the relief the fans are feeling after a result like that," he added.
"It was dangerous. Someone tried to take my boot off. People tried to kiss me and were biting me. It was scary."
Reacting to the events on Twitter, West Brom goalkeeper Ben Foster, who watched the game from the bench, bemoaned the "stupid scenes".
