Papiss Demba Cisse, second top scorer in the Bundesliga last season, scored for Freiburg early in the second half but turned villain when he missed the penalty nine minutes from time.
Lukas Podolski scored both goals to give Cologne a 2-0 win over Hannover 96 in Sunday's other game, denying their opponents a chance to go second behind leaders Bayern Munich.
Hamburg stayed bottom with seven points, but are now only behind Freiburg on goals scored and Augsburg on goal difference.
Arnesen, the club's sporting director, named himself as coach last Monday before the club managed to coax former Bayern Munich midfielder Fink from Swiss champions FC Basel on Thursday.
Fink will become the fourth coach to sit in the hot seat at Hamburg this season.
Michael Oenning quit last month after their poor start while interim coach Rodolfo Cardoso's provisional licence to lead a Bundesliga club expired after two league games in charge, prompting Arnesen to take over.
South Korea defender Heung-Min Son headed Hamburg in front after 12 minutes but Senegal forward Cisse levelled from close range two minutes into the second half after defender Slobadan Rajkovic misjudged a cross from the right.
Croatia forward Ivo Illicevic scored after a one-two with Gokhan Tore to put Hamburg back in front just after the hour.
Hamburg conceded a penalty when Jeffrey Bruma, having mis-kicked the ball, then tripped Stefan Reisinger. However, Cisse, who scored 22 goals last season and already has seven this term, fired his penalty high and wide.
"I'm very satisfied with the young players and the result," said Arnesen. "You must always make the most of your luck and luck was on our side."
Podolski opened the scoring against City when he curled a free-kick into corner in the first half and added the second near the end, bursting clear to rifle home an unstoppable shot from inside the area.
The result meant that Borussia Monchengladbach stayed second with 17 points, followed by defending champions Borussia Dortmund, VfB Stuttgart and Werder Bremen on 16 with Schalke 04 and Hannover on 15. Bayern Munich have 22 points.
