The 1991 European Cup winners parted company with 43-year-old Prosinecki following a 2-2 draw with neighbours Rad over the weekend and a row with the board.
Jankovic, who will remain in charge of the under-21 side, had a brief spell at Red Star during the 2007/08 season. His new club host Bordeaux in the first leg of a Europa League play-off on Thursday.
"It is an emotional moment for me to return to Red Star after four years and the fact that the club called me up in a difficult moment is very gratifying," Jankovic told a news conference on Tuesday.
"I am here because the Serbian Football Association showed plenty of understanding and allowed me to carry on with the Under-21 team, a group which I just couldn't abandon.
"Prosinecki is a close friend and it will take a lot of hard work to match his energy, dedication and commitment to Red Star."
Jankovic acknowledged that qualifying for the group stage of Europe's second-tier competition would be a tall order for Red Star.
"It's a very delicate moment and a massive task but I can't wait to get started," he said.
