An online petition asking the organizing committee to change its mind had garnered more than 30,000 supporters as of late Thursday night on change.org.
By way of contrast, a petition calling for the reinstatement of a conservation officer in B.C. who was fired for refusing to kill two orphaned bear cubs had more than 240,000 signatures.
The petition against West's appearance says the Games have triggered a sense of pride in the city, and the petition aims simply "to ask a proud Torontonian (or even a Canadian for that matter) to perform instead on July 26."
Among those suggested in the petition are Drake, Walk Off The Earth, Feist and Metric.
"Kanye West's words and conduct are antithetical to the spirit of the PanAm Games," one person wrote on the petition. "Canada abounds in musical talent that is undoubtedly better-suited for this event and should be featured at the closing ceremonies."
Not everyone agreed. News that West would appear at the Games was met with jubilation by some of his Toronto fans.
"PRAISE YEEZUS!" exalted one fan on Facebook in reference to West's last album.
Toronto rapper Maestro, who performed at the opening ceremonies, also noted that West is an international phenomenon.
"It would have been nice to see a Canadian but you know what, that's a staple for hip hop right there, so a shoutout to Kanye, a shoutout to Canada," he said on CBC TV.
The head of the Games organizing committee said it's too late to change the lineup — the contracts have already been signed through Live Nation, a concert promoter and Pan Am sponsor.
Saad Rafi defended the decision to book West, saying "major international-calibre events have international artists."
"Let's not forget, these are the Pan American Games, which include the Americas and the Caribbean. It's highly appropriate to have artists from those countries of origin," he said.
Rafi stressed that closing ceremonies are meant "predominantly for athletes" rather than the general public, adding that several athletes work out to West's music.
Mayor John Tory, meantime, made headlines by wishing aloud that organizers had chosen a Canadian for West's slot — but only after he had to concede that, until now, he'd somehow believed the iconic rapper and husband to reality superstar Kim Kardashian was Canadian.
On Thursday, Tory tweeted a minute-long video poking fun at his ignorance, saying: "Yeezus, can't believe I didn’t know @KimKardashian's husband wasn't Canadian!"
In the video, West's song "Stronger" plays as the straitlaced Tory grooves to the music, his head moving like a bobblehead doll, as he waits on a subway platform wearing a suit and tie with a red maple leaf hat, earphones and sunglasses.
He then takes a seat on the train and leafs through a newspaper until he comes to a hidden Rolling Stone magazine with the famously voluptuous Kardashian on the cover.
West will be joined at the July 26 show by Miami pop-rapper Pitbull and six-time Juno-winning singer Serena Ryder.
— Follow Paola Loriggio on Twitter @ploriggioSuggest a correction