The AIK midfielder struck in each half with two fine finishes to ensure the World Cup quarter-finalists made it five games without defeat under interim coach Paulo Wanchope.
Lee Dong-gook, the leading scorer in the K. League, cancelled out Costa Rica's opener on the stroke of half-time, but Borges restored the visitors' lead and Oscar Duarte added a third goal with 12 minutes remaining.
It was South Korea's first defeat under German coach Uli Stielike, who was confirmed as Hong Myung-bo's successor last month, while Wanchope has masterminded four wins out of five since taking temporary charge.
Borges opened the scoring with a clinical right-footed finish eight minutes before half-time after Bryan Ruiz had headed the ball down into his path.
Lee fired the hosts level in time added on at the end of the first half, getting in front of his marker to convert from close range after Son Heung-min had picked him out.
Costa Rica were not to be denied, though, and Borges restored their advantage when he turned in David Ramirez's centre with a cheeky back-heel two minutes after the break.
Duarte then gave the Central American side breathing space when Kim Seung-gyu flapped at a Ruiz corner and the defender made him pay by heading home.
Ki Sung-yueng had a late goal ruled out for offside and there was no way back for Stielike's men.
‘Maybe I’ll be signed by Wrexham, who knows? English football would suit me – if an offer came, I wouldn’t think twice, I’d go immediately’ Euro 2024 cult hero reveals his UK dream
‘Managing Leeds? It was an option that appeared, but it wasn’t the right timing. I decided it wasn’t a good idea to leave the club I was at mid-season’: Premier League boss admits to turning down opportunity to replace Jesse Marsch in 2023
