"CFR Cluj's club management and Ioan Andone have reached an agreement to end relations," the club said in a statement.
"We are proud of everything we've accomplished together. Andone has and will always have a special place in the hearts of our supporters."
According to media reports, former Napoli and Lazio coach Edoardo Reja is on a shortlist to replace Andone, who could rejoin Rapid - the club he coached in 2010.
"We'll announce the name of the new coach tomorrow," CFR Cluj's owner Arpad Paszkany told local media. "It's 99 percent certain that it will be a foreigner."
The Railwaymen, playing for more than an hour with 10 men, drew 1-1 at Galatasaray in a lively Champions League Group H match on Tuesday to stay second behind Manchester United having won one, drawn one and lost one of their opening three matches.
United have nine points, Cluj have four, followed by Braga (3) and Galatasaray (1).
However, local media and fans criticised Andone for CFR's domestic performances. They are 10th in the Romanian top flight with 16 points from 12 matches, 13 points behind leaders Steaua Bucharest and only six points above the relegation zone.
Much-travelled Andone, who led Cluj to a domestic double in 2008, began his second spell as coach in April after replacing Portuguese Jorge Costa and guided the team to their third league title.
Former international defender Andone previously coached another eight Romanian clubs, including Bucharest rivals Dinamo and Rapid, as well as being at the helm of teams in Cyprus, Saudi Arabia, United Arab Emirates and Bulgaria.
