The victory for the Russian champions, the first time Barca have lost a competitive game at home since a la Liga defeat to Osasuna last May when they had already won the Spanish title, leaves the group wide open at the halfway stage.
Inter Milan slipped to last place on three points after a 2-2 home draw with Dynamo Kiev and Barca, Rubin and Dynamo all have four points from their three games.
Rubin rocked Pep Guardiola's side in only the second minute at the Nou Camp when midfielder Ryazantsev let fly from distance and the ball arrowed past Victor Valdes into the top right corner.
Barca had plenty of possession after falling behind but found it tough to break down the visitors' well-organised defence until Xavi picked out striker Zlatan Ibrahimovic with a lofted pass in the 48th minute.
The Sweden striker controlled with his chest and crashed a low shot past Sergei Ryzhikov from the right of the area but Barca fell behind again after a swift Rubin break engineered by Alejandro Dominguez.
The Argentine forward released the pacy Karadeniz and the Turkish international finished with ease past Valdes with 17 minutes of normal time left.
WILD CELEBRATIONS
Guardiola threw on midfielder Seydou Keita in place of central defender Rafael Marquez with 10 minutes left and Barca laid siege to the Rubin goal but were unable to find a second equaliser.
Ibrahimovic sent an acrobatic volley against the crossbar, Xavi stung Ryzhikov's hands with a fierce drive and Yaya Toure headed against a post in the fourth minute of added time.
The final whistle prompted wild celebrations among the small knot of Rubin fans in the vast arena as the shell-shocked Barca faithfuls slipped quickly away.
Rubin coach Kurban Berdyev said his side's strategy had been to try to contain Barca and hit them on the break.
"The whole team played well and they showed great control during the match and that's why we won today," Berdyev told a news conference.
"I am very proud and I think today will be a big party for our fans," he added, before being applauded from the room by the Russian journalists.
Barca came into the match after Saturday's 0-0 La Liga draw at Valencia and Guardiola said they had missed a chance on Tuesday to all-but secure qualification for the knockout phase.
"These things happen. It's not the first time and I doubt it will be the last," he told a news conference.
"I didn't feel like we played a bad match. They had a very well-structured defence and all the teams (in the group) are very good. They are all professionals."
Barca next face a tricky away match against Rubin in Kazan before hosting Inter. They travel to Kiev to face Dynamo on December 9.
