Harper said Monday that the result of the so-called referendum is illegitimate and was conducted under an illegal Russian military occupation.
Harper said his government is putting economic sanctions and travel restrictions on senior people in Russia and Ukraine, and in Crimea specifically.
The Russian individuals include deputy prime minister Dmitry Olegovich Rogozin, Sergey Yur'yevich Glaz'yev, an adviser to Russian President Vladimir Putin, and Putin aide Vladislav Yur'yevich Surkov.
The Ukrainians include Serhiy Valeriyovich Aksyonov, the prime minister of Crimea and Volodymyr Andriyovych Konstantynov, chairman of the Crimean parliament.
Harper announced the sanctions before meeting Vadym Prystaiko, the Ukrainian ambassador to Canada.
"It's my strong belief we must keep the pressure on and we must continue to maintain sanctions and maintain putting in place strong steps to dissuade this behaviour," Harper said. "What the Putin regime has done cannot be tolerated and can never be accepted."
The prime minister said the sanctions are aimed at isolating Russia.
"The individuals targeted are responsible for undermining the sovereignty and territorial integrity of Ukraine and for facilitating Russian military action against Ukraine," he said.
He said the measures demonstrate to Russia that its actions have consequences.
"Canada will not stand by while Russia violates Ukrainian sovereignty and territorial integrity."
Harper is to fly to Kyiv on Friday to show his support for the temporary government of Ukraine in the face of Russian aggression in the strategic Black Sea region.
"I'm really looking forward to that as an opportunity this weekend to express the solidarity of the Canadian people with Ukrainian people and obviously our support for Ukraine's prosperity, its sovereignty and its territorial integrity."
Canada is acting in concert with its allies, Harper said.
Earlier Monday, President Barack Obama's administration announced a freeze on the U.S. assets of seven Russian officials, including top advisers to President Vladimir Putin.
European Union foreign ministers also imposed travel bans and froze the assets of 21 officials from Russia and Ukraine.
Harper condemned the "illegal, dangerous and unilateral actions of the Putin regime in Crimea on the weekend" when 97 per cent reportedly voted "yes" to joining Russia.
"They continue to seek a dangerous escalation in the situation," he said.
The U.S. administration has said there were "massive anomalies" in the ballots.
"The referendum in Crimea was a clear violation of Ukrainian Constitutions (sic) and international law and it will not be recognized by the international community," Obama said.
He said there are ways out of the crisis if Russia withdraws its troops and agrees to a dialogue with Ukraine.
He said the United States will work with its allies to support Ukraine politically and economically.
"And as we go forward, we'll continue to look at the range of ways we can help our Ukrainian friends achieve their universal rights and the security, prosperity and dignity that they deserve."
Also on HuffPost