The interim Liberal leader says Harper "lied" to Canadians during last May's election about having contractual protection against skyrocketing costs for the F-35 jets.
And he says Harper deliberately misled Parliament as well.
The prime minister is trying to calm the furor over a biting report by auditor general Michael Ferguson. The report concluded that the National Defence Department manipulated the process, low-balled cost estimates and kept Parliament in the dark to ensure it got the fighter jets it wanted, without competition.
Harper told the Commons Wednesday his government will follow suggestions from the auditor general on how to make the process for replacing Canada's CF-18 fleet more open and accountable.
"The auditor general has given a recommendation on re-examining cost estimates," Harper said.
"The government will do that."
Harper reiterated that the government believes now, as it did in 2010, that Canada is protected from significant increases in the price of the F-35 jets, if it decided to buy them.
"It is the case that it is the government of the United States that has agreed to cover escalation in the development costs (for the F-35)," Harper said.
Rae says it's simply not credible for Harper to claim he was unaware of the problems.
He notes that the news has been full of reports of escalating costs and that both the Congressional Budget Office in the U.S. and Canada's parliamentary budget officer, Kevin Page, have pegged the costs at billions more than the $9 billion touted by the Harper government.
"You can't get away with the fiction that a $10-billion mistake in calculating the cost of the F-35 stealth fighter ... had nothing to do with the man in charge, with the man whose name and whose moniker is on every single publication of this government," Rae told a Liberal caucus meeting that was thrown open to reporters.
"He cannot now pretend that he was just the piano player in the brothel who didn't have a clue as to what was really going on upstairs."
While he expects a public servant or maybe even a cabinet minister or two might get "thrown under a bus," Rae said: "A massive fraud of this kind can't be ascribed to anyone other than the prime minister of Canada.
"This is Stephen Harper's baby. ... That's why we say and I say without hesitation, Stephen Harper is not fit to be the prime minister of Canada."
Rae added: "I'll tell you who I think should resign. Stephen Harper should resign. The buck stops at the top."
NDP Leader Thomas Mulcair was more circumspect. He said he believes Defence Minister Peter MacKay is "ultimately responsible" for the mess but stopped short of calling for his head to roll.
"It's premature to ask for a resignation. We don't have all the details," Mulcair said.
In response to Tuesday's auditor general's report, the government has announced it will spend no more than $9 billion to purchase new fighter jets, give annual updates to Parliament on the program's progress and shift lead responsibility for procurement of the jets from Defence to the Public Works Department.
Also on HuffPost