Goals by Chinedu Obasi and Knowledge Musona either side of half-time saw the hosts come from a goal down after Cologne had taken a sixth minute lead through Mato Jajalo in the one-legged tie.
Champions Borussia Dortmund had little trouble disposing of second division's Dynamo Dresden, with goals from Robert Lewandowski and Mario Gotze, but other top tier clubs needed to dig deep to beat lower-ranked teams.
Hamburg SV looked to be on the brink of elimination against regional league club Eintracht Trier before defender Dennis Aogo curled in a free-kick for the winner in the second half of extra-time.
Germany international Aogo struck from about 22 metres in the 110th minute to give new coach Thorsten Fink his first taste of victory, minutes after Hamburg keeper Jaroslav Drobny had denied the hosts with a spectacular save.
Ahmed Kulabas had put Trier in the driving seat before Marcus Berg's 63rd minute equaliser sent the game into extra time.
Borussia Monchengladbach were made to work even harder against third division's Heidenheim needing penalties to advance.
Gladbach keeper Marc-Andre ter Stegen saved twice from the spot to send his team through 4-3 on penalties after a goalless 120 minutes.
Fellow Bundesliga club Augsburg needed a 62nd minute goal by Daniel Brinkmann to edge past fourth division team RB Leipzig.
Holders Schalke 04 travel to Karlsruher SC on Wednesday with league leaders Bayern Munich entertaining second division Ingolstadt.
