Galtier's side led 1-0 at half-time thanks to Romain Hamouma's 18th-minute opener but Toulouse took control of Friday's Ligue 1 clash in the half hour after the break with goals from Martin Braithwaite and Issiaga Sylla.
While Galtier was frustrated by his team's inability to convert a plethora of chances versus Toulouse, he was particularly riled by Saint-Etienne's poor defensive display to begin the second half.
Braithwaite tucked home the equaliser for Toulouse in the 63rd minute after Saint-Etienne goalkeeper Stephane Ruffier failed to hold onto Etienne Didot's drive and spilled the ball at the Danish striker's feet.
Toulouse tore Saint-Etienne's defence apart with a fine string of passes 12 minutes later, which saw Serge Aurier pick out Sylla at the back post, who tapped the ball home from close range.
"We both had totally different half-times," Galtier said.
"Those 30 minutes during the second half were terrible for us, against an opponent with their own qualities and who did not steal the victory."
Saint-Etienne had 18 shots to Toulouse's 11 at the Stade Geoffrey-Guichard but only put three on target, while the visitors managed six to test Ruffier.
Galtier's team led after 17 minutes when Hamouma poked the ball into the net, following a mistake by Toulouse goalkeeper Ali Ahamada but failed to score again, which cost Saint-Etienne at least a point.
"We made a great first half and were lucky enough to score first," Galtier said.
"We gave a lot of effort in the first half but sadly, we scored only once.
"Then, during the first 25 minutes of the second half, we were far from good.
"Toulouse played the way they know, like we expected them to before the game.
"We needed to be more accurate. We did not score when we had the chances."
The loss to Toulouse meant Saint-Etienne missed the opportunity to move above Monaco to the top of Ligue 1.
Galtier's side have 12 points from six games and remain second in Ligue 1, although they are likely to drop down the table with Paris Saint-Germain (11 points) and Marseille (10) having not yet played this weekend.
