But what about the senior cabinet minister from Quebec?
He says he's not too bothered by it.
Denis Lebel, who is the Conservatives' Quebec lieutenant, was asked in an interview Tuesday what it was that upset his government about the religion plan.
"There's nothing that upsets me in there," Lebel told The Canadian Press.
This is while his colleague Jason Kenney, the minister for multiculturalism, has posted a picture of himself on Twitter wearing a turban to protest the charter.
The government also says it will consult Justice Department lawyers should the PQ plan pass the legislature and, if necessary, will launch a legal challenge if it's deemed to violate minority rights.
But Lebel says it's too early to get exercised.
"We'll respect provincial jurisdiction, let people make their choice and, after that, we'll see — we'll make sure Canadians' rights are respected," Lebel said.
Asked whether the idea appeared constitutional Lebel replied: "I'm not a lawyer. Is this the actual text that they'll vote on in the national assembly? Nobody knows."
The PQ government has not actually tabled a bill yet. It promises to table one in the fall based on a preliminary plan it recently laid out.
The plan, in its current form as advertised, would forbid people with Sikh, Muslim and Jewish headwear from holding government jobs, along with Christians wearing larger-than-average-crosses.
Since the current plan has insufficient support from the legislature's other parties, the PQ has two basic options: water it down in the bill, or stick to its guns and run an election on the idea.
So far the PQ has shown little desire to water it down. In fact, the government has hinted it might even toughen the plan by narrowing its five-year opt-out clauses for institutions.
Although several polls suggest the plan has considerable support in Quebec, there has been vigorous opposition and many prominent Quebecers have spoken out against it.
The pro-charter side is pushing back.
A former Supreme Court judge has joined the pro-charter cause. A pro-secularism group that held its first event Tuesday said its 60 members include retired justice Claire L'Heureux-Dube.
Some members of the Rassemblement pour la laicite held a news conference, without the ex-judge present, to explain their support for the PQ plan.
The group's supporters include L'Heureux-Dube, former student strike leader Martine Desjardins, union leaders, and members of minority communities who oppose religious accommodation.
Leila Bensalem, an Algerian-born teacher at a multiethnic school in Montreal, applauded the PQ initiative. She said the list of demands for religious accommodation at school is growing, from halal food to separate gym classes for boys and girls.
"We are literally bombarded, day after day and week after week, by demands for reasonable accommodations," she said Tuesday.
"We've been asking ourselves for years, 'When will someone finally stand up and put an end to these hot potatoes in schools?' "
She said religious clothing is the first way fundamentalists begin to exert their influence.
"That veil is an ideology... The fundamentalists, when they establish an Islamic republic, the first thing they ask from Muslim women is to wear the veil," Bensalem said.
"It's like the flag they want their women to wear. They represent fundamentalism... Even if they say, and they keep saying, 'It's my choice,' they forget to say this is the only choice they have."
L'Heureux-Dube was not at Tuesday's event. However, in an interview this year with Radio-Canada, she expressed concern about women covering their faces.
The judge also argued that while certain rights are fundamental, like the right to life and equality, other civil liberties "can be reduced" in a "free and democratic society."
L'Heureux-Dube, 86, was appointed to the Supreme Court by ex-prime minister Brian Mulroney in 1987. She was the second woman ever named to the high court and she served until 2002.
The question of whether the PQ plan would ever pass the constitutional test has been a matter of some contention.
The PQ says its plan was studied, and endorsed, by provincial Justice Department analysts, although there have been reports that the government ignored legal advice it didn't like.
The government is also under pressure to show any feasibility studies for its plan — such as the scope of the problem it's supposed to be responding to, and what the impact would be on people and institutions.
Attempts to reach L'Heureux-Dube this week were unsuccessful, and the Rassemblement pour la laicite said she was travelling outside the country.
-With files from Alexander Panetta
Also on HuffPost