With Chile trailing 1-0, Bielsa took off his out-of-touch playmaker Matias Fernandez after 32 minutes and within four minutes of the move saw his team go 2-1 up.
Chile, whose last World Cup appearance was in 1998, have 30 points from 17 games and are guaranteed a top four finish in the South American group with one match to play. The defeat meant that Colombia, eighth with 20 points, were eliminated.
"Matias is a player who connects the start of the move with the attack and (Jorge) Valdivia is a player who participates fully in the attack, we needed a midfielder who was less of a midfielder and more of a forward," said Bielsa.
Colombia, who have managed a measly 12 goals, went ahead after 13 minutes when Chile midfielder Arturo Vidal turned Jackson Martinez's cross into his own net.
Three minutes later, Waldo Ponce headed Chile level and one minute after that Humberto Suazo met Fabian Orellana's low cross with a diving header to put Chile in front.
Valdivia clipped the ball past David Ospina to regain the lead for Chile in the 72nd minute and repaid Bielsa's faith in him seven minutes later by setting up Orellana for the fourth.
