Roberto Trashorras squeezed his 91st-minute spot kick under Diego Lopez for a 1-0 victory on the night, after home defender Angel was adjudged to have tripped Pedro Botelho as he raced into the area.
Celta, who drew 1-1 at home in the first leg, moved into next week's quarter-finals where they will meet Atletico Madrid or second division Recreativo Huelva.
Recre, who hold a 3-0 first-leg lead, visit the Calderon on Thursday.
"I was a long way from the penalty and I don't know if it was inside or outside the area, or whether he touched the player or not, but I'm not interested now because we are out," Villarreal coach Ernesto Valverde told reporters.
"I don't want to look for excuses. You can't call their qualification for the quarters unfair."
Celta are 14th in the second division and beset with financial problems, but there was little to separate the two sides in a half-empty stadium.
Villarreal appeared to have settled for the 0-0 draw that would take them through as the second half drew to a close, when Angel slid in to tackle Botelho on the edge of the area and conceded the decisive penalty.
Holders Barcelona need to overturn a 2-1 deficit when they visit Sevilla on Wednesday in the last-16 round's highest- profile tie.
