The Newcastle United manager caused widespread outrage when he thrust his head into the face of Hull City midfielder David Meyler during his side's 4-1 victory on Saturday.
The incident followed a brief shoving exchange on the touchline as the Irishman prepared to take a throw-in.
Newcastle fined Pardew £100,000 and the Football Association have confirmed they will be looking into the incident, while Bevan conceded that there was no defending Pardew's actions.
"The buck stops with Alan," he told BBC Radio Five Live. "It's unacceptable, it's inappropriate and it's insupportable from every perspective and Alan knows that.
"He immediately realised the serious error, (made) sincere apologies to all parties and obviously (has) deep regret.
"It was good to see (Hull manager) Steve Bruce's reaction and Hull accepting (Pardew's apology).
"But Alan does need to think hard about how not to put himself in that position again."
Bevan offered warm praise to Newcastle for the club's handling of the matter, describing their actions as "swift" and "professional".
"I was pleased to see Newcastle in a very short period of time making a very swift, professional response that provided Alan with a very heavy fine and a formal warning," he added.
"In the communication I've had with Alan, he's extremely sorry regarding the incident and we're going to talk later today (Sunday) about it as well.
"It shows you the strains even when a manager's team is in a winning position, but it didn't take Alan half a second to realise it was unacceptable."
The LMA chief also revealed that his organisation have been looking into the possibility of making changes to technical areas in English football.
He explained: "We did a technical report six or seven months ago, interviewing 40 referees and 40 managers, and we're looking at the moment how the technical area works in America, for example, in other sports and seeing how we can look to improve several problems that occur because of the positioning.
"What we can do is make a serious effort to look at how the technical area should be placed."
