The Edinburgh outfit have been gradually clawing back the deficit - handed down for entering administration - since the start of the season, but have now reached positive figures after 24 matches.
Callum Paterson netted the opener with a left-footed effort in the 37th minute, before Yann Songo'o levelled matters on the stroke of half-time.
Scott Robinson settled the encounter, firing home from the centre of the penalty area in the 80th minute to seal Hearts' first win in 12 matches.
The Tynecastle side remain bottom of the Scottish Premiership on two points, 19 adrift of 11th-placed Partick Thistle.
At the top end of the table, Aberdeen held Motherwell to a 2-2 draw - a result that will play into the hands of leaders Celtic, who play on Sunday.
Stephen McManus put the hosts ahead at Fir Park with a 47th-minute header, before Adam Rooney nodded Aberdeen level 19 minutes later.
But that joy was shortlived as Motherwell retook the lead through Zaine Francis-Angol's strike from close range two minutes later.
Russell Anderson had the last word though, scoring with a left-footed effort in the fourth minute of injury time.
Victory for Celtic at Hibernian on Sunday would see them move 15 points clear at the top of the league, with two games in hand over second-placed Aberdeen and one over third-placed Motherwell.
Elsewhere, Kilmarnock recorded a 2-0 win over Inverness Caledonian Thistle, who now have just one victory to show from their last seven fixtures, while St Mirren and Partick played out a goalless draw.
