Stoke took the lead through Oussama Assaidi at St James’ Park on Boxing Day, but then saw Glenn Whelan and Marc Wilson sent off within three minutes of each other prior to half-time.
Despite seeing Loic Remy have a penalty saved following Wilson's dismissal, Newcastle duly ran riot to secure a seventh win in nine games.
Remy atoned for his error from 12 yards with two goals, while Yoan Gouffran, Yohan Cabaye and Papiss Cisse also found the net as Stoke were predictably over-run.
A disappointed Hughes said: "I think the referee needs to look at his performance. From one of our more senior referees, I was absolutely dismayed by his performance if I’m honest.
"Up to the point when the first sending off was given I thought we were totally in control of the game.
"We were 1-0 to the good; Newcastle were finding it very difficult to really have an impact on the game and we had a number of other chances.
"Then the referee makes a couple of decisions which change the course of the game."
In addition to criticising Atkinson for both red cards, Hughes pointed to a handball prior to Newcastle's equaliser through Remy and also believed the ball had gone out of play in the build-up to the hosts' second goal, scored by Gouffran.
We save the penalty (that followed Wilson's dismissal). We go down the other end with nine men and we have an opportunity to play somebody in for a clear opportunity on goal," added Hughes.
"It gets handballed by (Mike) Williamson, the referee is two yards away, plays on and they go up the other end and score. So you can understand why we're a little bit aggrieved by what's happened.
Asked whether he felt the ball had gone out before Gouffran put Newcastle ahead, Hughes replied: "Yes, in my view. But at that point we'd come to expect key decisions not to be given correctly."
Hughes was also sent to the stands by Atkinson after Whelan had been sent off, but had no complaints on that score.
"That's because there was a ball near me and I needed to get rid of some frustration so I booted it on the pitch. It was totally the wrong thing to do so I apologise for that," explained the Welshman.
