Blackburn's victory, which included two Arsenal own goals, helped them climb off the bottom of the table while all three promoted sides won on the same day for the first time since February 2010.
Swansea City - who failed to score in their opening four matches - won for the first time, beating West Bromwich Albion 3-0 at the Liberty Stadium with Scott Sinclair scoring their first goal in the top flight since 1983 with a first-half penalty.
Queens Park Rangers won 3-0 at Wolverhampton Wanderers with new skipper Joey Barton grabbing the opener while Norwich City won their first game of the season 2-1 at 10-man Bolton Wanderers even though Norwich conceded a penalty for the fifth successive league match.
In the day's other matches Everton came from behind to beat Wigan Athletic 3-1 at Goodison Park while Aston Villa drew 1-1 with Newcastle United at Villa Park.
EWOOD THRILLER
The day's opening match produced a seven-goal thriller with Blackburn twice coming from behind to beat Arsenal as Aiyegbeni Yakubu scored twice on his Rovers debut and Alex Song and Laurent Koscielny put through their own net to give Rovers the points.
Arsenal, returning to the north west for the first time since being beaten 8-2 by Manchester United three weeks ago, took the lead when Gervinho scored his first goal for the club after 10 minutes.
Yakubu equalised 15 minutes later before his former Everton team-mate Mikel Arteta, who scored for Everton at Ewood Park three weeks ago, put Arsenal back in front after 34 minutes with a rising shot from the edge of the box.
Blackburn were level again after 50 minutes when Arsenal failed to clear a free-kick and the ball bounced off Song's thigh and past goalkeeper Wojciech Szczesny for the equaliser.
Nine minutes later Yakubu scored from point-blank range and was ruled onside to make it 3-2 and 10 minutes after that a stunning break from Blackburn which involved Junior Hoilett and substitute Martin Olsson ended with Olsson's cutback being turned into his own goal by Koscielny to make it 4-2.
Substitute Marouane Chamakh scored with a superb header to give Arsenal hope with five minutes remaining but Rovers survived a late onslaught to ease the pressure on manager Steve Kean after 250 fans staged a protest march before the game calling for his sacking.
Wenger told Sky Sports he was concerned by his team's start to the season where they have taken just four out of a possible 15 points.
"If you want to win football games you have to be focused for every minute, and that cost us today, it was very frustrating because if you look at the statistics we should have won easily," he said.
Kean told Sky: "Today we showed a lot of character which everyone has been questioning, well not everyone, just one per cent of people, about 200 who demonstrated and hopefully we can turn them around because the rest of the people who are behindus got a fantastic performance today and that's exactly what we are all about."
SWANS TAKEOFF
While emotions were running high at Blackburn, they were in Swansea too where a minute's silence was held before the start to mark the death of manager Brendan Rodgers' father and this week's tragedy at the nearby Gleision Colliery which claimed the lives of four local miners.
The mood was lifted after 14 minutes when Sinclair opened the scoring from the spot after Paul Scharner brought down Joe Allen and Swansea were well worth the three points after a 24th minute header from Leroy Lita and a third early in the second half from Nathan Dyer.
QPR also enjoyed a 3-0 win thanks to goals from Barton, Alejandro Faurlin and substitute DJ Campbell in a match Wolves probably wished had not gone ahead.
The match was in doubt until 90 minutes before kick-off because of a power cut in the city centre, but electricity was restored in time for the game to be played as Wolves failed to score for the third successive time.
