"I wanted to wish everyone a really, really Merry Christmas, Happy Hanukkah … all you infidel atheists out there, I want to wish you the very best also," he said.
"I don't know what you celebrate during the holiday season. I, myself, celebrate the birth of Christ."
Pallister made the remarks to Natalie Pollock, a former cable TV host who now posts videos to her Pollock And Pollock News Channel on YouTube.
University of Manitoba political studies professor Royce Koop said his impression of the video is that it was a joke gone bad.
"The joke didn't really seem to go over too well. It had kind of an awkward dad quality to it," he said.
Koop thinks Pallister should consider clearing the air, adding politicians don't usually use that kind of language.
“Some people wish that politicians would be somewhat less guarded, that they would joke around. Unfortunately, when politicians do joke around, they can sometimes be condemned very quickly and it cannot go well for them.”
Manitoba Liberal leader Rana Bokhari said even if Pallister was joking, it wasn't funny.
"We did hear phone calls and emails regarding that but you know what, I'm going to hope that it was a joke. But if it was a joke, it was in poor taste," she said.
"No leader of a political party — actually no leader — should be using such divisive language."
Pallister not joking
Pallister on Monday defended his message, saying it was not a joke or meant to be funny. Rather, he was trying to be inclusive.
He said he used the word infidel in the truest sense, pointing out it means non-believer, so he was trying to appeal to everyone.
He said the message is being torqued.
A secular association, the Humanists, Atheists and Agnostics of Manitoba, is disappointed with the comments and surprised the leader of a political party would say such a thing.
"For those of us who don't believe [in religion], it came off as kind of awkward and rather a little condescending and it just seemed to me that the more he spoke, the more awkward it got," said Donna Harris, a spokeswoman for the group.
Harris has extended an invitation to Pallister to educate him on what she calls non-believers.
"You would expect better from someone who's the leader of a political party in our province, to have a bit more knowledge and understanding about the various groups that make up our very diverse population. And it's not like we're an insignificant part," she said, noting that 15 to 20 per cent of Manitobans identify themselves as non-believers.