The voters and the Council of Canadians, which bankrolled the Federal Court suit, say an appeal wouldn't uncover who was responsible for the fraudulent calls.
"I feel we have taken this case as far as we can go," said Peggy Walsh Craig of North Bay, Ont., one of the applicants in the case.
Federal Court Judge Richard Mosley concluded earlier this month that fraud was a factor in the misleading calls, but not enough of one to justify overturning the result.
The ruling cleared the Conservative party and its candidates of any effort to mislead voters, but said the most likely source of information used to make the calls was the party's voter database, accessed by a person or persons unknown.
But the judge wouldn't go so far as to overturn the results.
"The scale of the fraud has to be kept in perspective,'' Mosley wrote in the decision.
"The number and location of the complaints received by Elections Canada from across Canada indicates that the voter suppression effort was geographically widespread but, apart from Guelph (Ont.), thinly scattered."
The ruling left both sides in the dispute — the Conservatives in one corner, the voters who fielded the calls in the other — claiming victory of a sort.
Six ridings were involved: Vancouver Island North in British Columbia; Yukon; Saskatoon-Rosetown-Biggar in Saskatchewan; Elmwood-Transcona and Winnipeg South Centre in Manitoba; and Nipissing-Timiskaming in Ontario.
Craig said she joined the suit because she was upset.
"I was appalled that someone tried to manipulate the May 2011 federal election in my riding," she said at a news conference.
"I feel completely vindicated by the ruling."
Garry Neil, executive director of the Council of Canadians, said the ruling shows that fraud was widespread.
"At least one per cent of eligible voters did not vote in 2011 because they received one of these fraudulent calls," Neil said. The council still wants to find out who was responsible and ensure it doesn't happen again, he added.
Elections Canada is also investigating fraudulent robocalls, stemming from complaints that have surfaced in 56 ridings across the country