Despite a patchy 2013-14 season, Hart has started in each of City's three Premier League matches so far.
Pellegrini, whose side face Arsenal on Saturday, said Caballero would soon get his chance.
The Chilean tactician wants to create competition for the number one role, and that could mean Caballero – who joined City from Malaga in July – starts at Arsenal.
"I'm not waiting for Joe Hart to make a mistake to change him for Willy Caballero," Pellegrini said.
"Maybe in the last year Joe didn't start very well but he finished very well. But Willy is a very good goalkeeper.
"If you want to be a top goalkeeper you must have all the attributes, good distribution, to know when to go for the ball, to be good one-on-one and to not concede early goals.
"Willy has a good career behind him. It's important to have competition in every position in the team, not just with Joe Hart and Willy Caballero.
"Replacing Joe Hart with Willy Caballero doesn't make a difference.
"Tomorrow we'll play against Arsenal and one hour and 15 minutes before kick-off you will know which goalkeeper starts."
City have conceded just twice in three league matches this season, winning two of those encounters.
