Serbian Danijel Ljuboja gave Nice the lead after 30 minutes but Reims hit back immediately through Romain Amalfitano and then equalised for a second time when Odair Fortes cancelled out forward Mouloungui's effort just before the hour mark.
However, Gabon's Mouloungi had the final say when he scored the winner seven minutes from the end of extra-time.
Reims, famous for losing two European Cup finals in 1956 and 1959 to Real Madrid before fading into obscurity, eliminated Ligue 1 title challengers Stade Rennes in the last round.
Cup 'shocks' are a regular occurrence in France but this season the surprise results have reached new heights.
Amateurs Chambery, the first fifth division team to knock out three top-flight sides, host second tier Angers on Wednesday when Ligue 1 leaders Lille host FC Lorient in their quarter-final.
Paris Saint-Germain also welcome second division Le Mans.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.