Ginczek struck the decisive goal in the first minute of second-half stoppage-time at the Mercedes Benz Arena, capping off a dramatic finale, which included a red card to Stuttgart forward Martin Harnik and an equaliser from Bremen with four minutes left.
But as the clock ticked past 90 minutes in Stuttgart, the home side launched a counter-attack, which saw Geoffroy Serey Die burst through the middle, before sliding a pass to Ginczek and the striker finished between the legs of Bremen goalkeeper Raphael Wolf.
Stuttgart (26 points) sit 17th in one of two automatic relegation positions, just ahead of Hamburg (25), with Paderborn (27) in 16th; a spot that entails playing a two-legged play-off with Bundesliga 2's third side in order to stay in the top division.
"We put in a huge performance," Ginczek told Stuttgart's website.
"Even with a man down and after conceding that late equaliser we still went all out for the win – and the winner was just pure emotion. We had that little bit of luck on our side in the end.
"We're still in a relegation place but we're determined to battle our way out of trouble. That's what we're all working towards and everyone has a part to play."
Stuttgart were reeled in twice before Ginczek finally gave them victory with Christian Gentner's drive on the quarter-hour mark cancelled out by a header from Davie Selke four minutes into the second half.
Coach Stevens hailed Stuttgart's "courage" to keep attacking, especially after a raft of misses and Harnik's second yellow card two minutes prior to Vestegaard's header.
"We should have done better with our chances in both halves and won it comfortably really, so we only have ourselves to blame in that regard," the 61-year-old coach said.
"The team showed a lot of courage and in the end they got the necessary stroke of luck. But we worked for it and that was a result of a concerted effort."
