PEN announced Sunday that the writers were upset by Charlie Hebdo's portrayals of Muslims and "the disenfranchised generally."
The Paris-based magazine, where 12 people were killed in a January attack at its offices, is to receive a Freedom of Expression Courage Award at the May 5 event in Manhattan. Much of the literary community rallied behind Charlie Hebdo after the shootings, but some have expressed unhappiness with its scathing cartoons of the Prophet Muhammad and other Muslims.
Francine Prose, a former PEN American president, has also withdrawn.
"I was quite upset as soon as I heard about (the award)," Prose told The Associated Press during a telephone interview Sunday night. Prose said she was in favour of "freedom of speech without limitations" and that she "deplored" the January shootings, but added that giving an award signified "admiration and respect" for the honoree's work.
"I couldn't imagine being in the audience when they have a standing ovation for Charlie Hebdo," Prose said.
The gala is the highlight of PEN's annual, week-long World Voices Festival and is intended as a celebration of artistic achievement and expression, with past award winners including Ondaatje, Salman Rushdie and Philip Roth. Besides Charlie Hebdo, which will be represented by editor in chief Gerard Biard and critic and essayist Jean-Baptiste Thoret, others receiving awards include playwright Tom Stoppard, Azerbaijani journalist Khadija Ismayilova and Penguin Random House CEO Markus Dohle.
Prose and Ondaatje were among more than 60 writers scheduled to serve as hosts. According to PEN, the other hosts who decided not to attend were Teju Cole, Rachel Kushner, Taiye Selasi and Peter Carey.
In a letter sent earlier Sunday to PEN trustees, current PEN American president Andrew Solomon acknowledged that several people were offended by some of Charlie Hebdo's cartoons, but added that PEN believed strongly in the "appropriateness" of the award.
"It is undoubtedly true that in addition to provoking violent threats from extremists, the Hebdo cartoons offended some other Muslims, as their cartoons offended members of the many other groups they targeted," Solomon wrote.
"But, based on their own statements, we believe that Charlie Hebdo's intent was not to ostracize or insult Muslims, but rather to reject forcefully the efforts of a small minority to place broad categories of speech off limits, no matter the purpose, intent or import of the expression," he said. "We do not believe that any of us must endorse the contents of Charlie Hebdo's cartoons in order to affirm the principles for which they stand, or applaud the staff's bravery in holding fast to those values in the face of life and death threats."
The Charlie Hebdo protest is the biggest controversy for the PEN American Center in recent memory. In 1986, Norman Mailer infuriated many writers when he invited then-Secretary of State George P. Shultz to address the annual Congress of International PEN. E.L. Doctorow complained at the time that Mailer, the PEN American president, was turning the gathering into "a forum for the Reagan administration."
ALSO ON HUFFPOST: