Coach Mano Menezes is still struggling to find the ideal line-up for the 2014 World Cup hosts but Marcelo gave them a fourth-minute lead.
Vedad Ibisevic levelled for Bosnia nine minutes later following blunders by defender David Luiz and goalkeeper Julio Cesar.
Both teams missed chances until Papac turned the ball into his own goal in the 91st minute as he tried to clear Hulk's powerful low cross.
Ronaldinho, who made his international comeback against Ghana last year, was given another start but the 31-year-old was a shadow of his former self and was substituted in the second half.
The game appeared anything but a friendly with the Bosnia side playing as if the match was one of the most important of their careers.
Although it was staged in a Swiss second division stadium, the 17,500-capacity arena was packed with fanatical Bosnian fans who drowned out the Brazilian national anthem, jeered incessantly when the opposition had possession and threw flares on the pitch.
"The role of the supporters is to make conditions difficult for the opposition but it should never be done in excess," said Menezes who was also unhappy with some crunching tackles.
"Neymar suffered a criminal foul the first time he got the ball. That wasn't football."
Five-times world champions Brazil had plenty of the ball but rarely threatened while Bosnia were always menacing on the break.
NEW STYLE
In 18 months in charge Menezes has drastically changed Brazil's playing style, moving them away from the physical, counter-attacking game favoured by predecessor Dunga to a more patient, passing approach.
However, it is still not clear whether his players are up to the task especially with the added pressure of knowing the next World Cup will be on home soil.
Brazil, who had won their previous five games, went ahead with a simple goal as Dani Alves threaded a pass through a flat defence into the path of Marcelo and the Real Madrid player beat goalkeeper Asmir Begovic with a low shot into the far corner.
Luiz, criticised for his performance in Chelsea's 3-1 defeat by Napoli in the Champions League last week, then lost the ball to Edin Dzeko who fed Ibisevic and his shot slipped through Julio Cesar's hands.
"We kept making mistakes with our final pass, we funnelled our moves down the middle and that left us vulnerable to the counter-attack," said Menezes.
Miralem Pjanic was denied by Julio Cesar just before half-time and Zvjezdan Misimovic blasted over after Dzeko, a constant threat, had pulled the ball back.
The second half was a similar story although Brazil improved markedly when Paulo Henrique Ganso replaced Ronaldinho.
Neymar had their best effort when he twisted inside his marker and just failed to curl his shot around Begovic.
Dzeko had several good openings and could have grabbed a late winner for Bosnia when he burst through
