A Kieran Gibbs own goal and Wayne Rooney's strike five minutes from time were enough to render Olivier Giroud's injury-time effort a consolation.
But United were indebted to goalkeeper David de Gea for keeping them in a first half that Arsenal completely dominated.
Van Gaal identified slack passing as the root of his team's early problems and was also critical of United lacking ruthlessness when they improved after the break.
"I was not pleased with the way we played in the first half but, of course, it was an away game against a top team and we have won," said Van Gaal after watching United triumph on the road for the first time in his tenure and move into fourth place.
"It was the right moment to win this game but I have to say that the first 35 minutes, we gave the ball away so easily - that is not possible for a top team.
"Because of that Arsenal created a lot of chances. Fortunately we have a very good goalkeeper and after the first 35 minutes you saw that we were coming back into the game.
"We created a few possibilities and at half-time I said that when we keep the ball and show confidence then we shall create many, many chances.
"We created at least three big chances - [Angel] Di Maria could have finished the game - but also two times where the [final] pass was easy to give to Di Maria and Wayne Rooney and the pass did not arrive.
"Those were not very difficult passes in my opinion so we have to improve that."
United's struggles during the opening stages were compounded by an ankle injury to Luke Shaw - further stretching defensive resources that forced Van Gaal to field rookie centre-backs Tyler Blackett and Paddy McNair in a 3-5-2 formation.
"I hope there are players coming back from injuries because it is unbelievably difficult to create consistency in the team," the Dutchman added.
"Every week I have to change my line-up. I am the kind of coach who always has the same line-up and maybe changes one or two - not constantly [changing] so much players.
"Not the players but also the formation. That is not always good for the team."
