Striker Guillermo Beltran got Cerro Porteno off to superb start after only 10 minutes before Angel Romero continued their momentum going into half-time with a 43rd minute strike.
Olimpia fought back in the second-half, pulling a goal back through Diego Centurion but they could not find an equaliser.
The hard-earned victory means Cerro Porteno remain unbeaten and four points clear of second-placed General Diaz, who came from behind to defeat Rubio Nu 2-1.
Diaz had a terrible start, going behind after six minutes to Francisco Vera's strike.
But Diaz regrouped in the second-half, equalising through Diego Doldan in the 65th minute before winning the game with a penalty from Cristian Martinez four minutes from time.
Nacional Asuncion secured their first win of the season with a 5-2 triumph against Deportivo Carapegua.
Asuncion claimed a dream start, as they scored in the first minute from Cesar Caceres Canete's penalty before Alfredo Mazacotte doubled the lead on 9 minutes.
When Canete scored his second goal in the 16th minute the game was effectively over despite Cristian Ayala pulling a goal back for Carapegua on the stroke of half-time.
Asuncion added two more goals in the second-half through Juan Arguello and Marcos Prieto while Ruben Caceras scored a consolation goal for Carapegua.
In other results, Guarani drew 2-2 with Sportivo Luqueno, Deportivo Capiata won 2-0 over Libertad and Sol de America defeated Cerro Porteno PF 2-0.
‘Scoring in a World Cup is like winning the title – can you imagine millions of people celebrating something you did? It’s insane and made me very proud’: Brazil legend explains how much 2002 goal meant to him
'He already had the ability but didn’t use it because he was afraid to shoot – I said, "You’re a player who has to decide games – you have to take risks, mate"': Liverpool star was forced to become confident by team-mates
