Harper reinforced the message of a formal statement issued earlier Wednesday by the G7 during an address to a business group in Vancouver.
"All of the G7 countries remain collectively, strongly committed to the view that we will not accept Russia's illegal occupation of Crimea," the prime minister said after returning from Seoul where he had just announced a new free trade deal with South Korea.
The G7, which includes some of the world's top economies, said a Russian-backed referendum on the status of the Black Sea peninsula would have no legal effect.
The statement also said the referendum process is flawed by its rushed nature and the deployment of Russians troops in the Crimea.
"We call on the Russian Federation to immediately to halt actions supporting a referendum on the territory of Crimea regarding its status, in direct violation of the Constitution of Ukraine," it says.
"Any such referendum would have no legal effect. Given the lack of adequate preparation and the intimidating presence of Russian troops, it would also be a deeply flawed process which would have no moral force. For all these reasons, we would not recognize the outcome."
The G7 countries — Canada, France, Germany, Italy, Japan, the United Kingdom and the United States — were joined in the statement by the president of the European Council and the president of the European Commission.
They said a Russian takeover of Crimea would violate a number of international agreements.
"In addition to its impact on the unity, sovereignty and territorial integrity of Ukraine, the annexation of Crimea could have grave implications for the legal order that protects the unity and sovereignty of all states," the statement says.
"Should the Russian Federation take such a step, we will take further action, individually and collectively."
The G7 urged Russia to withdraw its troops and offered to mediate talks between Moscow and the Ukrainian government.
NATO, meanwhile, is flexing its muscles in the region.
The military alliance deployed two surveillance planes to monitor Ukraine's air space as well as ship traffic in the Black Sea as Russia's military buildup in the region continued.
A NATO spokesman says the two aircraft will monitor Russian movements from inside Poland and Romania — two NATO members that border Ukraine.
Ukraine and Russia traded barbs Wednesday over Moscow's military movements near Ukraine's border.
The Ukrainians said the buildup raises the threat of an invasion, but Russia denied that.
Andriy Parubiy, secretary of Ukraine's national security and defence council, told reporters in Kyiv that Russia has deployed more than 80,000 troops, up to 270 tanks and 140 combat planes close to the border, creating the "threat of a full-scale invasion from various directions."
In Moscow, Deputy Defence Minister Anatoly Antonov denied a military buildup along the nearly 2,000-kilometre border.
— With files from the Associated Press
Also on HuffPost