Enner Valencia made his first start for West Ham, but the World Cup star missed the crucial penalty in the shootout, allowing Michael Doyle to seal a 5-4 win following a 1-1 draw after 120 minutes.
Nigel Clough led his League One side to wins over two Premier League teams on their run to the FA Cup semi-finals last term and they showed their giant-killing instincts again on Tuesday to doggedly take the game the distance.
In front of a packed Upton Park crowd, Valencia looked a threat from the off through the right channel, stinging Mark Howard's palms from 25 yards 20 minutes in before the former Arsenal trainee stood tall to deny the Ecuador striker in a one-on-one.
The visitors were again indebted to Howard when he tipped Ravel Morrison's free-kick around the post and then impressively kept out teenage debutant Reece Burke's acrobatic effort from the resulting corner.
Allardyce's side looked slack immediately after the break, however, and Sheffield United took advantage as Bob Harris zipped up the left flank and his cross was flicked on by Mark McNulty, giving a hapless Reid no chance to react, shinning into his own net.
Mauro Zarate was introduced as the hosts grew desperate, the Argentine attacker bringing a scrambled save out of Howard after the keeper had kept out Downing's powerful effort from range, but it was not enough and the game drifted into extra time.
Howard again made a stunning save to deny Mark Noble's curler and the hosts failed to craft a telling chance in the final 10 minutes, taking the game to penalties.
After eight perfect spot-kicks, Valencia stepped up for West Ham's fifth, only for Howard to save to his right.
