The criticism, from Martin Kallen, UEFA's director of the championship Poland will stage with Ukraine in 14 months' time, comes days after the latest hooligan outbreak when Polish fans wreaked havoc at a friendly with Lithuania in Kaunas on Friday.
Kallen, organising his fifth European Championship for UEFA, said none had caused as many problems as this tournament and was highly critical of Poland's football image.
Kallen said: "We are concerned but I know the Polish government is also concerned but we are not going to let a minority spoil it for the majority.
"They know they have a huge image problem, there are always hooligans around every match day in the league but the government is making the right steps for the future.
"What we saw at last Friday's match was not a very good picture to see that happening in a stadium. We and the Polish government are concerned, they are taking this seriously and will have changes in the next month."
He added: "For the Euro, different people will be coming to matches -- there will be more families. The Euros are a party and in many areas there is more a problem on a daily basis for club matches. But, clearly, we are facing some challenges.
Marcin Herra, head of the Poland 2012 organisers, told delegates there would be "zero tolerance" for hooligans but he was quick to add that the championship would leave a lasting positive legacy for his country and Ukraine.
"We have new legislation which allows us to work much more precisely against those hooligans.
"There will be zero tolerance to make sure that 500 people cannot spoil the event for one million people."
