The audit made nine recommendations, including that the agency develop a new official languages policy, create closer ties with French and English minority communities, recruit bilingual returning officers in all 338 ridings and commit to bilingual services at all polling stations during operating hours.
There were more than 15,000 central polling stations during the last general federal election.
"We're very aware of the challenges that would represent," said Commissioner Graham Fraser, who launched the audit after receiving 26 language complaints about the last general election.
Elections Canada argued in its responses to the audit that it was "neither practical nor necessary" to require returning officers to be bilingual in all 338 federal riding, and that the level of service should be determined through demographic analysis.
"We cannot guarantee in-person service to the public of equal quality in English and in French at all times in all polling places irrespective of whether the demands for such service exist," Elections Canada wrote.
Elections Canada did say it's committed to providing equal services in French and English where there is "significant demand," including the National Capital Region.
The audit, conducted between June and August 2014, found that returning officers were not always aware of official language minority communities in their districts. During a visit to a returning officer's office in Toronto, an auditor was told there were no Francophone communities in the district despite census data detailing the opposite.
Still, Fraser said he was pleased that Elections Canada accepted most of the recommendations, and noted the agency produces a high quality bilingual website, publications and ballots.
'Standard too low'
Elections Canada is "setting the standard too low," said Denis Vaillancourt of L'Assemblée de la francophonie de l'Ontario, which advocates on behalf of Francophones across the province.
Even in Eastern Ontario, where there are nearly 250,000 Francophones, some are unable to be served in their first language at the polls, he said.
"It's an inconsistent picture and it's sometimes frustrating," he said. "Many of our member groups could help in recruiting bilingual workers at the poll stations and help alleviate that issue."
James Shea said his anglophone wife is uncomfortable going to vote alone in Quebec.
"The comfort level she has in going to vote is, she goes with me in the event we're looking for direction. I'm bilingual so we're able to address it in French or in English," he said.
Shea, president of the Regional Association of West Quebecers, said the federal government has a responsibility to make sure voters are not intimidated out of going to the polls because they only speak one official language.
Democratic Reform Minister Pierre Poilievre said Friday he had not yet read the audit but that it was Elections Canada's job to offer bilingual services.
"Every single Canadian voter had the right to be served in the official language of their choice and it's up to Elections Canada to make it so," he said.