The minority PQ government is threatening to revoke the town's bilingual status if the English-speaking population falls below 50 per cent.
In a recent EKOS poll commisioned by CBC, 84 per cent of respondents said if the PQ were to win a majority, it would do more to limit the English language.
Residents of Lac-Brome, also known as Brome Lake, are seeing the effects of the government's stance on language.
Right now, city signs, street names, even parking tickets are written in both French and English in the town.
Official bilingualism is not permitted in most of Quebec, but the town was exempted from the rules because of its English heritage.
Gilles Decelles has been the town's mayor since the 1980s, back when the English-French ratio was 75 per cent to 25 per cent.
But he said when he last checked, the total anglophone population had dropped to 47 per cent.
"There's no question the anglophones feel insecure by the election of the Parti Québécois," Decelles said.
He said it wouldn't surprise him if some English-speaking residents have considered moving since the election.
"If you start discarding things like bilingual signs ... it's another signal to the English community that, 'Hey, we want you assimilated.'"
Jean-François Lisée, the minister responsible for anglophones, said the Parti Québécois government does not intend to alienate English speakers.
"There is no willingness to assimilate or trample on the rights of Anglo-Quebecers," he said.
"Our task is to gain more integration of allophones."
Suzanne Demers, a Lac-Brome business owner, says the area was settled by English families, and their roots go back hundreds of years.
"How can you just strip that away?" she wonders.
While she is passionate about being a Quebecer, Demers said she is also proud of being Canadian.
"If I can't have that status in my own province, then yes, I'm leaving," she said.