The hosts had conceded just one goal in six previous Premier League games at St Mary's Stadium this season but were torn apart by the champions, even when City had gone down to 10 men.
Southampton survived a clear penalty appeal when Sergio Aguero was fouled by Jose Fonte early on and held out until half-time.
However, they fell behind to a deflected Yaya Toure strike on 51 minutes and conceded further goals to Frank Lampard and Gael Clichy after Eliaquim Mangala had been sent off.
"I think the start of the second half was better than our start in the first half, but if you make mistakes, you will get punished for that," said Southampton manager Koeman.
"Even when it was 11 against 10, we made mistakes and they scored two goals more - that's not good.
"We knew we had to be good to get a result. We defended well early on, they didn't create a lot of opportunities.
"But when I thought we could do a little more pressing in the second half, we went behind and it was not good enough."
Southampton's cause was not helped by having to replace Morgan Schneiderlin at the interval due to a thigh injury.
"Schneiderlin is an important player for us and we don't have enough quality players to keep making changes like they [City] can," added Koeman.
"Maybe the first goal was also a little unlucky because it took a bit of a deflection, but it was not good enough today.
"If I look at my team, maybe three players reached their level - Toby Alderweireld was fantastic.
"But it's difficult to create chances against a good team like Man City. We knew they were a good team and they showed that."
