"The last few months have shown that many young players have stepped into the spotlight and have good prospects," Low explained.
"After several phone conversations with Michael it is now the time, before the start of the European Championships (next year), to have a clear position."
Ballack, who won 98 caps and scored 42 goals since making his debut in 1999, has not played for the national side since March last year after sustaining an ankle injury that forced him to miss last year's World Cup.
One of the midfielder's lowest career moments came when, having steered Germany into the 2002 World Cup final, the midfielder was forced to sit out the showpiece match through suspension, with Brazil running out eventual 2-0 winners and lifting the trophy.
"Michael Ballack was a leading player in the team for a decade and he has a large share in the team's success since the 2002 World Cup," Low added.
Low said he had repeatedly told the player, who was retained as the official captain until now, that the younger generation of players, who were part of Germany's youngest World Cup squad in 76 years last year, were the future.
"In our talks I had the impression that he understood our view. So in the interest of everyone an honest and clear decision was appropriate," Low said.
"We very much regret this decision because in our view Michael Ballack could still be an important part of the national team," club CEO Wolfgang Holzhaeuser said. "But that is the decision the coach took."
