Their performance might not have matched their FA Cup upset win over the Gunners in the 1978 final but the Championship side, who sacked manager Roy Keane last week, can dream of a Wembley return thanks to a 78th-minute goal by Hungary striker Tamas Priskin.
Arsenal manager Arsene Wenger, without a trophy since 2005, decided this was too good an opportunity to miss and fielded just about his strongest available side.
Cesc Fabregas was, as ever, at the heart of most of Arsenal's probings but the Premier League title-chasers were guilty of their usual sin of over-elaboration and struggled to carve out clear chances.
Fabregas did set up Andrei Arshavin early in the second half but the Russian shot badly wide with just keeper Marton Fulop to beat.
At the other end Connor Wickham, a burly 17-year-old forward, gave Arsenal's defenders a tough time and must have had Wenger wondering if a muscular presence such as his could be the missing link that would complete his neat-passing jigsaw.
Fellow front man Priskin found the net with an overhead kick but the goal was ruled out for offside.
FASCINATING RETURN LEG
The longer the game went on the more Ipswich grew in confidence and, having lost in the semi-finals twice in the 1980s, their fans began to wonder if this could be third time lucky.
That result began to look possible 12 minutes from time when Ipswich's pounding of Arsenal's soft centre finally paid off.
Colin Healy played a long ball that caught out Johan Djourou and Priskin ran on to slot the ball past Wojciech Szczesny.
Arsenal increased the pressure in the dying minutes but could not find an equaliser, setting up a fascinating return leg at Emirates Stadium on January 25.
On Monday Ipswich announced that Paul Jewell would replace Keane he left the managerial duties to caretaker Ian McParland on Wednesday.
"It shows character to come back from the result against Chelsea but we could have had three goals I am absolutely dead proud of them," McParland said.
"I have done my bit tonight and my aim was to keep them in the tie and if they go to the Emirates and work hard they will be a tough prospect.
"I will probably be moving to pastures new and hopefully I will be managing a club again soon. I have had a great time here, there are good people and good fans and it won't take much to turn it around."
Wenger had few complaints after a flat display. "With playing so many games you could see tonight that we had no spark," said the Frenchman.
"Ipswich were sharper and we made a mistake which they took advantage of. The good thing is that we play at home in the second leg and that can make a difference. We will not always be as lacking in creativity as we were tonight."
In the first leg of the other semi-final, played on Tuesday, West Ham United beat Birmingham City 2-1 at Upton Park.
