In a statement issued following a rare emergency cabinet meeting held Saturday afternoon, Harper said Canada is recalling its ambassador from Moscow and pulling out of the G8 process being chaired by Russia.
At the same time, Canada is recognizing the legitimacy of the new, pro-Western Ukrainian government.
"Ukraine's sovereign territory must be respected and the Ukrainian people must be free to determine their own future," Harper said in his statement. "We call on President Putin to immediately withdraw his forces to their bases and refrain from further provocative and dangerous actions."
He warns that should Putin push forward with military action, it will lead to "ongoing negative consequences" for Canada-Russia relations.
The special cabinet meeting was called after Russian parliament gave Putin the green light on Saturday to use the country's military in Ukraine. Russian armed forces have seized control of Ukraine's Crimean Peninsula.
Canadian cabinet ministers met by phone and in person well into the evening on Saturday, agreeing to join the United States and the United Kingdom in issuing strongly worded condemnations of the Russian actions and taking concrete steps to show they mean business.
"We join our allies in condemning in the strongest terms President Putin's military intervention in Ukraine. These actions are a clear violation of Ukraine's sovereignty and territorial integrity. They are also in violation of Russia's obligations under international law," Harper stated.
U.S. President Barack Obama and Harper spoke by telephone about the troubling events in Ukraine earlier Saturday. The two leaders agreed to "co-ordinate closely."
Obama personally delivered the same message as Harper's during a 90-minute telephone conversation with Putin, the White House said late Saturday. He not only urged Putin to pull his forces back to Russian bases in Crimea, but also to refrain from interfering elsewhere in the former Soviet republic.
The Kremlin said Putin, meantime, told Obama that Russian citizens living in Ukraine are in danger and that Russia has the right to protect them.
"Vladimir Putin emphasized that, in the case of a further spread in violence in eastern regions (of Ukraine) and Crimea, Russia maintains the right to protect its interests and the Russian-speaking population that lives there," the Kremlin said in a statement.
Political turmoil in Ukraine pushed President Viktor Yanukovych from office following massive pro-democracy protests after he rejected a partnership agreement with the European Union in favour of deepening his country's historical ties with Moscow.
Yanukovych held a news conference in southern Russia on Friday in which he said he was not asking Moscow for military assistance and called military action "unacceptable."
Yanukovych, who still considers himself Ukraine's president, also vowed to "keep fighting for the future of Ukraine" and blamed the U.S. and the West for encouraging the rebellion that forced him to flee last weekend.
Ukraine's acting president, meantime, Oleksandr V. Turchynov, said at a briefing in Kyiv that he had ordered Ukraine’s armed forces "to full combat readiness." Ukrainian soldiers have also reportedly been ordered to open fire if attacked by Russian troops or others.
Canada will boycott meetings leading up to the G8 summit planned for Sochi later this year, Harper said Saturday. The G8 host changes every year, and 2014 is Russia's turn. The G8 host normally holds several planning meetings among finance ministers, foreign ministers and other top officials.
However, Canadian athletes will still compete in the upcoming Paralympics in Sochi, the Prime Minister's Office said late Saturday. The games are set for March 7 to 16th.
Harper said Canada supports the United Nations sending international monitors to Ukraine and is also involved in multilateral talks to put together a financial aid package for the beleaguered country.
"We will continue to cooperate closely with our G7 partners and other allies. Should President Putin continue on this course of action, it will lead to ongoing negative consequences for our bilateral relationship," Harper stated.
In Canada, the federal NDP said it was deeply worried about the tension in Ukraine.
"We are monitoring events closely and with great concern. We urge the Canadian government to continue to work alongside our allies towards a peaceful resolution of this troubling situation," spokesman Marc-Andre Viau said in an email.
The Department of Foreign Affairs, meanwhile, is advising against travel to the region and says Canadians already there should consider leaving "while it is safe to do so."
Foreign Affairs says its advisory is due to "political uncertainty" and reports of armed groups operating in the area, along with protests and public clashes.
Harper's statement comes as a day after Foreign Affairs Minister John Baird and a Canadian delegation visited Kyiv to show support for the new government.
But Canada did not officially recognize the legitimacy of that government until Harper's statement on Saturday night.
Serhy Yekelchyk, a professor of Slavic studies at the University of Victoria, praised Ottawa's response as "strong" and "timely" but said it might not be enough.
"I think Canada can go further, especially given the special connection we have to Ukraine, with 1.2 million Canadians being of Ukrainian background," he said in an interview.
"If this doesn't work, there will be an issue of further sanctions," such as excluding Russia from future G8 meetings, he said.
"Because if this goes unchecked, it is very easy to see Putin interfering with other former Soviet republics."
With files from The Associated Press
Also on HuffPost