Dwight Gayle and Marouane Chamakh both found the net at The Hawthorns on Saturday to send Palace into the fourth round for the first time in four seasons.
However, with so few people in the stands, Pulis believes the competition may be in danger of losing some of its magic.
"There's got to be some way of filling stadiums up, especially for third round and fourth round ties at Premier League grounds," he said. "That has to be looked at because it is a wonderful competition.
"For all of what is going on, we've go to try as a nation to make sure we keep it a great competition by filling stadiums for games - that's certainly a help.
"It's difficult because it's just after new year, people have spent a lot of money, so you understand that.
"But, as I said, there's got to be a way at this time of year that clubs can attract full houses, how they do it you'd have to ask chief executives."
With attendances poor at grounds across the country, including a showing of just 6,960 as Wigan Athletic took on MK Dons, West Brom boss Keith Downing insisted the club had made an effort to tempt fans to the stadium.
"The club cut the prices so they tried to get people in," he said.
"But this area is still a difficult area, it's still trying to survive economically, and it's tough for people to keep on producing money to watch football."
