Their statement, published Thursday in the Globe and Mail and La Presse newspapers, comes as the Conservative government proposes a new, expanded mandate for the Canadian Security Intelligence Service to counter terrorist threats.
"Protecting human rights and protecting public safety are complementary objectives, but experience has shown that serious human rights abuses can occur in the name of maintaining national security," the statement says.
"Given the secrecy around national security activities, abuses can go undetected and without remedy.
"This results not only in devastating personal consequences for the individuals, but a profoundly negative impact on Canada's reputation as a rights-respecting nation."
The Security Intelligence Review Committee currently oversees CSIS, doing several studies each year and tabling a report in Parliament.
Critics argue the review committee is just that, a review body, not an oversight agency peering over the spy service's shoulder in real time.
The joint statement published Thursday was signed by Jean Chretien, Paul Martin, Joe Clark, John Turner and 18 others involved in security matters over the years.
They include five former Supreme Court justices, seven former Liberal solicitors general and ministers of justice, three past members of the intelligence review committee, two former privacy commissioners and a retired RCMP watchdog.
They note that detailed recommendations for a new intelligence watchdog regime — put forward in 2006 by the federal inquiry into the Maher Arar torture affair — were not implemented.
Efforts to enhance parliamentary oversight of national security agencies have also been unsuccessful, they point out.
Several groups including Amnesty International, the Canadian Civil Liberties Association and the National Council of Canadian Muslims welcomed the statement.
The government's recently tabled anti-terrorism bill, which would give CSIS the power to disrupt plots, was debated Thursday at second reading in the House of Commons.
Opposition MPs accused the government of rushing the bill through Parliament. They said the new powers would allow security agencies to go after the government's enemies, such as environmentalists.
Justice Minister Peter MacKay dismissed the notion, telling the Commons that security forces would be better able to protect Canadians "in targeted and practical ways" with the review committee and the courts providing a check on their powers.
Speaking at an event in Surrey, B.C., Prime Minister Stephen Harper categorically rejected the idea of a security-cleared committee of parliamentarians monitoring spy agencies, like the ones in Britain and the United States.
"The model we have in Canada of independent, expert oversight — that's the model we're pursuing," he said. "We're going further in that direction, and we as a government are not interested in politicians doing the oversight."
The New Democrats chided the government for not doing enough to prevent the radicalization of young people, saying community engagement plans have been left to languish on the drawing board for years.
At a defence forum Thursday, security analyst David Perry of the Conference of Defence Associations Institute said that since 2007, CSIS has handed back five per cent of its total appropriation, or $180 million.
The RCMP had returned eight per cent, or $1.7 billion, to government coffers, Perry said.
New Democrat MP Jack Harris said the government was under-funding key security agencies in order to balance the books.
"They're bound and determined, come hell or high water, to have a budget surplus against all common sense and neglecting the security needs of the country."
Follow @JimBronskill on Twitter
Also on HuffPost