Clemente, 61, will be presented on Tuesday afternoon, the club said in a statement.
He replaces Inaki Tejada who was promoted following the sacking of Manuel Preciado two weeks ago.
The north-coast club gave Tejada the job until the end of the season but one draw and a 4-0 defeat by Valencia on Sunday prompted a quick change.
Gijon are 19th in the 20-team standings with 19 points from 22 games and have only won once since the turn of the year.
The outspoken Clemente has a vast range of experience having managed clubs such as Athletic Bilbao, a team he led to two league titles in the 1980s, Atletico Madrid and Espanyol.
He has also coached Serbia and Cameroon.
Clemente was sacked by Cameroon in October after they failed to qualify for the Africa Cup of Nations.
His first game at Gijon will be at home to Atletico in La Liga on Sunday.
Get the best features, fun and footballing frolics straight to your inbox every week.
Thank you for signing up to Four Four Two. You will receive a verification email shortly.
There was a problem. Please refresh the page and try again.