The victory was the Basque club's third of the season with last Saturday's 3-1 defeat to the Spanish capital giants their only blemish so far.
It was a nightmare start for Bilbao with a strong penalty claim denied before defender Erik Moran carelessly gave the ball away in central midfield in the 14th minute, allowing Celta de Vigo attackers Rafinha and Charles to pounce and link up, with the latter finishing clinically.
Celta went to sleep only four minutes later, however, with Bilbao delivering the ball in a deep from a set piece and after the defenders failed to clear, Mikel San Jose smashed home the equaliser.
The visitors had a golden opportunity to retake the lead on 26 minutes but Charles blazed his penalty over the crossbar, after Bilbao goalkeeper Iago Herrerin brought down Nolito in the area.
They were made to pay 15 minutes into the second-half as a lovely backheel by Iker Munian found Andoni Iraola in space and the full-back placed his finish past the keeper to put Bilbao into the lead.
Bilbao extended their advantage a few minutes later as San Jose's long-ball put Benat Extebarria through on goal and with the composure to probe in and cut inside, the central midfielder curled his finish into the opposite corner from eight yards.
Celta did manage to make the last ten minutes interesting through substitute Santi Mina's header but they could not find an equaliser as they recorded their first loss of the campaign.
Bilbao move up into fifth position, three points behind joint-leaders Barcelona and Atletico Madrid, while Celta drop to ninth-place.
In Monday's other clash, Elche and Real Valladolid fought out a 0-0 draw that keeps both in the bottom half of the standings.
