Cisse and Mathieu called up by France
Karim Benzema, Franck Ribery, Blaise Matuidi and Bacary Sagna have all pulled out of the squad named last week.
Forward Kevin Gameiro and defender Eric Adibal are also doubtful for the Albania game on Friday so Lazio striker Cisse and uncapped Valencia utility player Mathieu have been drafted in.
Gameiro and Abidal will stay with the squad in a bid to be fit for next Tuesday's final qualifier with Bosnia at the Stade de France.
France lead Group D by a point from Bosnia.