Tripolis trumped Mainz 3-1 in Greece to claim a 3-2 aggregate victory in the third round of Europa League qualifying, as the Bundesliga club failed to reach the group stage of the competition for the third straight attempt.
Mainz had previously lost to Romania's Gaz Metan Medias in qualifying for the 2011-12 Europa League, while they fell to Sevilla in the 2005-06 campaign.
But Thursday's loss appeared even harder to take for Mainz, as they were leading on away goals in Tripoli until Pablo Mazza struck his second goal of the game for the hosts with just four minutes remaining.
"This is pure disappointment but it's our own fault," Baumgartlinger said.
"We can't just give away the game like that."
Mainz won the first leg 1-0 last week thanks to Shinji Okazaki's goal but Tripolis levelled the tie in the 30th minute of the second leg when Pablo De Blasis' cross looped over goalkeeper Loris Karius and into the net.
But in the 86th minute, Mainz failed to defend a long ball, which was flicked on by Khalifa Sankare to Mazza, who converted from a tight angle, and suddenly Kasper Hjulmand's side were eliminated.
"We knew that the beginning of the season was going to be hard because of all the departures," midfielder Johannes Geis said.
