Harper issued a statement that offered "condolences to the people of Venezuela," but not the family of the flamboyant 58-year-old leftist leader, who died Tuesday after a long battle with cancer.
A statement from a senior Venezuelan government official says a "card of protest" was sent to Ottawa after Harper expressed what he called insensitivity at a time when their country is grieving.
A wordy note from the vice-minister for North America, Claudia Salerno, said Caracas was protesting "in a blunt and categorical way, the statements issued the 5 of March 2013 by the prime minister of Canada, Stephen Harper, as they constitute insensitive and impertinent sentiments at a time when the Venezuelan people are grieving and crying over the irreparable physical loss of the Commander President Hugo Chavez Frias."
Harper said in his short statement on Tuesday that he hopes the death of Chavez brings a more promising future for the Venezuelan people.
"At this key juncture, I hope the people of Venezuela can now build for themselves a better, brighter future based on the principles of freedom, democracy, the rule of law and respect for human rights," Harper said in a statement Tuesday evening.
Harper also said that he looked forward "to working with (Chavez's) successor and other leaders in the region to build a hemisphere that is more prosperous, secure and democratic."
Harper has in the past pointedly challenged the world view of the influential Venezuelan leader, notably in a lengthy one-on-one interview with the Postmedia news service nearly four years ago before he was about to meet Chavez at the Summit of the Americas.
Harper had said Chavez was emblematic of the leftist leaders in the Western Hemisphere who were "opposed to basically sound economic policies, want to go back to Cold War socialism ... want to turn back the clock on the democratic progress that's been made in the hemisphere."
The Venezuelan government fired back on Wednesday saying that it "has freely and democratically chosen its Socialist destiny, is obliged to remind the representative of the Canadian government, that it has been thanks to this Bolivarian Revolution that our future as an independent and sovereign country appears more radiant and promising than ever, by virtue of the legacy of our historic leader, the Commander President Hugo Chavez Frias."
The statement from Caracas affirmed a commitment to "direct itself freely and with sovereignty towards Bolivarian Socialism and recognizes it as the way toward a future of well-being, that will secure the greatest amount of happiness for everyone."
Chavez was a vocal opponent of the free market economics of Canada and the United States.
He led a leftist revival across Latin America that posed a direct challenge to U.S. influence in the region.
While Chavez introduced social programs that helped feed and house his people during his 14 years in power, his economy has sputtered.
Though Venezuela is an oil rich country, it lacks the capital to maximize its oil output and has been wracked by inflation.
Hundreds of thousands of tearful supporters carried their dead president's coffin through streets of Caracas on Wednesday in an epic farewell to their iconic president known simply as "our commander."
One of Chavez's closest socialist allies, Bolivian President Evo Morales, choked back tears and declared: "Chavez is more alive than ever."
Harper's remarks echoed those of the Obama administration in Washington.
The White House said in a statement that Washington supported the "Venezuelan people and its interest in developing a constructive relationship with the Venezuelan government."
"As Venezuela begins a new chapter in its history, the United States remains committed to policies that promote democratic principles, the rule of law, and respect for human rights," the statement read.
Also on HuffPost