Having spent the 2012-13 season in Argentina's second tier before earning promotion to the Primera Division, Olimpo had started the 2013-14 campaign poorly with three losses and two draws but broke through for their maiden win on Sunday.
Martin Perez Guedes scored twice for Olimpo as the Bahia Blanca-based club triumphed over visiting Boca.
The hosts took the lead in the seventh minute when Boca's defence failed to effectively clear Paulo Rosales' cross and the ball was sent back into the penalty area, where Guedes finished from close range after a shot from one of his teammates was deflected into the right winger's path.
Olimpo clinched victory in the second half with two goals from set pieces.
Striker Augustin Vuletich doubled Olimpo's lead with 20 minutes remaining when he got on the end of Damian Musto's flicked header from a corner to volley home at the back post.
Guedes scored the home side's third goal, getting his head to Ivan Furios' flick, following a free-kick from Rosales.
The win has taken Olimpo from second-last to 17th in the 20-team Primera Division with five points, while Boca sit 11th with nine points after their third defeat in five matches.
League leaders Argentinos Juniors stretched their unbeaten run to five games with a 1-0 win over All Boys.
Osvaldo Miranda scored the only goal of the game in the 29th minute when Hernan Boyero touched Rodrigo Gomez's cross into the path of his striking partner to finish from close range.
Argentinos Juniors have 13 points from six matches and sit one point ahead of Arsenal and San Lorenzo, who both won over the weekend.
Left-winger Martin Rolle scored the decisive goal for Arsenal as they won 1-0 away to Quilmes, while Martin Cauteruccio and Hector Villalba struck either side of half-time as San Lorenzo also triumphed on the road, overcoming Rosario Central 2-0.
River Plate defeated Tigre 3-0, Racing Club drew 1-1 with Lanus and Atletico Rafaela trumped Gimnasia La Plata 2-1.
In other results, Velez Sarsfield won 3-0 over Newell's Old Boys, Estudiantes drew 1-1 with Belgrano and Godoy Cruz defeated Colon 1-0.
