Steve Bruce's team had surged into a 2-0 lead when the teams met at Wembley in last season's final before the north London club secured a 3-2 extra-time triumph to lift the famous trophy.
But Hull were unable to worry Arsenal in the same manner during the opening stages this time, and the hosts were good value for Mertesacker’s headed opener.
Alexis, Nacho Monreal and the returning Theo Walcott all went close to doubling the lead before the early inventiveness faded from Arsenal’s play.
Arsene Wenger's men still largely retained control of the contest and Alexis sealed a deserved return to winning ways eight minutes from time, following their Premier League reverse at Southampton on New Year's Day.
Walcott returned to the Arsenal starting line-up for the first time since he tore knee ligaments in last year's third-round win over Tottenham, and the England man was joined by Alexis and Joel Campbell in a pacy forward line.
Arsenal immediately looked to press home their advantage, with Paul McShane brilliantly clearing off the line after Alexis had rounded Harper, and the goalkeeper diverted Monreal's near-post header from the resulting corner onto his near post.
The forward came up short once more in the 61st minute, pulling a left-footed shot wide when confronted by a sprawling McShane.
Harper scrambled out to save from Alexis in the 74th minute, the former Barcelona forward losing his footing at the crucial moment, before substitute Alex Oxlade-Chamberlain failed to convert Alexis' drilled cross.
The Chile star got the goal his efforts deserved with time running out, collecting Cazorla's pass and swivelling on the edge of the box to curl a low strike past Harper.
The result retains Arsenal's hopes of becoming the first team since Chelsea in 2010 to retain the FA Cup.
