Blatter said Friday that relegation sanctions for clubs are "not a simple solution."
"This will lead to people coming to the stadium wanting to stop the game intentionally," Blatter said at a FIFA-sponsored conference on ethics in sports.
In January, Blatter discussed sanctions for racism in an interview for FIFA's website, and said "the best would be the deduction of points and the relegation of a team, because finally the club is responsible for their spectators."
In his speech Friday, he insisted "drastic sanctions" were necessary, but questioned how far football authorities could go to deter racism.
"We have to do something," Blatter later told reporters. "But the danger is if we say the match will be replayed, or there will be a deduction of points, or whatever, this can open the door to groups of, let's say, bad hooligans to create these problems."
Blatter has appointed FIFA vice-president Jeffrey Webb to lead a task force studying football's problems with discrimination on the field and in stadiums, and eventually propose a code of sanctions.
Webb should present an interim report to FIFA's 209 member countries in Mauritius on May 31.
His team includes Kevin-Prince Boateng, the Ghana and AC Milan midfielder who fuelled debate on how football dealt with racist abuse by walking off the field in January and forcing a friendly against a fourth-tier Italian club to be abandoned.
Blatter, who met with Boateng in Zurich last month after the player addressed a United Nations anti-racism panel in Geneva, said the global players' union, FIFPro, urged applying uniform disciplinary sanctions worldwide.
"It must be the same standard, there will be a resolution to the congress," Blatter said.