The government says the move will affect 31 Christian and 18 non-Christian chaplains who had part-time contracts.
Full-time chaplains will still be available to provide spiritual advice to the general prison population.
A spokeswoman for Public Safety Minister Vic Toews says the government funds some full-time chaplains that are determined based on the number of inmates requesting services from each faith.
"Convicted criminals will continue to have access to religious services of their choice on a voluntary basis," Julie Carmichael said in an email.
"The government funds some full-time chaplains that are determined based on the number of inmates requesting services from each faith. These chaplains will also make themselves available to provide services to the general population."
But the Liberals and NDP are accusing the Conservatives of cutting the contracts of non-Christian chaplains — a claim the Tories deny.
"To me, this is an unbelievable government. They make a big fuss about the fact that they're creating an office of religious freedom in the department of Foreign Affairs and in Canadian prisons, they say, 'Well sorry, you have to make do with whatever is on offer and it's all spiritual, so that's fine,'" said interim Liberal Leader Bob Rae.
"So I'm sorry, this doesn't meet the mark and I find it offensive. I think it shows an incredible insensitivity towards religious diversity in the country and, you know, if they're interested in saving money, they can cut back on limos. They don't have to cut back on chaplains."
"This is not a costly program," said New Democrat MP Paul Dewar. "The minister has no justification for cutting it."
Conservative MP Candice Bergen says the military has run its chaplaincy program the same way for years.
"If it is good enough for our armed forces, then it is good enough for inmates in our federal penitentiaries," she said.
Early last month, the Correction Service of Canada put out a request for a proposal for a Wiccan chaplain who would provide about 17 hours of service a month in British Columbia.
Shortly after The Canadian Press reported on the contract in September, Toews' office issued a statement saying it would not proceed with the plan until after a review.