After a goalless first half, Adam Szalai silenced the home crowd in the 50th minute as the Hungarians snapped up a loose ball and went on the counter-attack.
Finland goalkeeper Jussi Jaaskelainen, who had come out of international retirement at the request of coach Stuart Baxter, rushed out of his goal but Szalai rounded him easily to roll the ball home.
The Finns upped the pace with the introduction of Jari Litmanen on 70 minutes and striker Mikael Forssell threw his side a lifeline when he equalised with four minutes to go.
The Finns threw everything forward in search of a winner and Shefki Kuqi had a glorious chance to take all the points for the home side but shot wide, much to the annoyance of the better-placed Litmanen.
Finland pressed forward but once again a careless pass allowed the Hungarians to break. They moved the ball quickly up the field and Dzsudzsak coolly finished the move, shooting low past Jaaskelainen to claim the three points.
It was Finland's third defeat in as many games and they sit alongside San Marino with no points at the foot of the table. Hungary's campaign is much healthier with three wins from four games.
