Mahmoud Elkadri, a mosque board member, said he noticed the vandalism when he went to open the doors for prayers early Friday morning.
He said his family has lived in Cold Lake since 1996 and has always felt welcome.
"You see a message 'Go home' — I just want to let everybody know, this is our home," Elkadri said.
"My kids saw it and they started crying and I said: 'Why are you crying?' They said: 'We were born here and raised here and this is our home.'"
Cold Lake is home to a large military base where six Canadian CF-18 fighter jets departed Tuesday to join the international mission combating Islamic extremists in Iraq.
By Friday afternoon residents of the city, population 14,000, had posted colourful signs on the front of the mosque saying "You are Home!" and "We stand united as Canadians."
Then people started showing up with ladders and buckets to clean the paint off.
"Cold Lake showed its true stripes today," a proud mayor Craig Copeland told The Canadian Press Friday afternoon.
"Cold Lakers came out and supported the Muslim community and by 3:30 in the afternoon the windows were replaced and the graffiti was gone. This is what makes Cold Lake a great place to live."
Cold Lake citizens came together and cleaned the mosque that was vandalized pic.twitter.com/7odEsBkWo8— Peter Lozinski (@ptlozinsk) October 24, 2014
"I am very sure whoever did this could be just a drunk, random person at night time or it could be ... someone not from Cold Lake," he said. "We are very well acquainted in the community of Cold Lake."
Elkadri said the mosque has been open for four years at its current location and has been targeted before, but it was part of vandalism done to several businesses in the area.
RCMP Sgt. Josee Valiquette said police are not investigating the case as a hate crime at this point.
"We go where the facts take us. It's vandalism at this point," she said. "It could be anybody for a motive or no motive at all."
Multiculturalism Minister Jason Kenney tweeted: "As Multiculturalism Minister, I condemn the hateful vandalism @ Cold Lake Mosque. Thanks to local residents for their heart-warming response.
— By Tim Cook in Edmonton; with files from radio station CJXK
Note to readers: This is a corrected story. An earlier version misspelled Elkadri
ALSO ON HUFFPOST: