The Premier League side needed an 88th minute goal from captain Thomas Vermaelen to force the extra period but failed to make their superiority tell and it was Vermaelen who missed their final spot-kick to send the minnows through.
Garry Thompson had scored with a tidy far post finish in the 16th minute to give a surprise lead to the hosts who then held off a second-half barrage before Arsenal's Belgium centre back Vermaelen levelled with a far-post header.
"We had many, many chances, their keeper had a great game," Arsenal's under pressure manager Arsene Wenger told Sky Sports. "We couldn't convert our chances. When you go out it's never good enough for Arsenal.
"I think we lost today on penalties against a team who defended well. I cannot fault the effort of our team, we gave absolutely everything for 120 minutes.
"We have to give credit to Bradford on the night. That's part of the game as well. We couldn't convert our chances for different reasons. It was a very tough game. We'll get over that it's part of our job," he added.
In the night's other quarter-final, Andreas Weimann scored two late goals as Aston Villa booked a place in the last four by coming from behind to beat Norwich City 4-1 away.
TERRIBLE START
Arsenal, without a trophy since winning the FA Cup in 2005, selected a strong team but got off to the worst possible start.
Thompson put the hosts ahead with a tidy side-foot finish at the far post after Will Atkinson climbed high to flick on a Gary Jones free-kick from the right.
Having ruffled Arsenal's feathers, Bradford enjoyed a period of pressure before the visitors gained a foothold and began to look dangerous with Francis Coquelin firing a drive against the woodwork in the 38th and Gervinho guilty of a terrible miss.
The Ivory Coast striker somehow failed to make proper contact with the ball from three metres with Bradford keeper Matt Duke beaten and the goal gaping.
If the luck was against the Premier League side on that occasion, it was firmly in their favour when Nahki Wells almost doubled Bradford's lead with an early shot that beat keeper Wojciech Szczesny but trickled just wide of the post.
Arsenal got on top in the second half and pegged their opponents back but struggled to create chances against a stubborn back-line.
Substitute Alex Oxlade-Chamberlain forced a good save from Duke with Arsenal's first effort on target in the 70th minute, before Vermaelen rose at the far post to head home a cross from Santi Cazorla with three minutes remaining.
SHOOTOUT KINGS
Arsenal piled on the pressure in extra time but a second goal was elusive and the match went to penalties, with Bradford full of confidence having won their last eight shootouts.
Arsenal's Spain midfielder Santi Cazorla and Morocco striker Marouane Chamakh missed with the visitors' first two spot-kicks.
Bradford let them b
