A spokesman for Foreign Affairs Minister John Baird says it's incredibly disappointing that the AU chose a brutal dictator to lead it.
The 90-year-old Mugabe, who has ruled Zimbabwe since 1980, was appointed to the top post of the 54-nation AU during its two-day summit in Ethiopia.
Mugabe has been accused of serious human rights violations, including using violence to win elections.
The economy of his once-thriving country has plummeted since his government started seizing white-owned farms 15 years ago.
The country suffered hyper-inflation until it abandoned its currency for the U.S. dollar in 2009.
"Canada strongly values its relationships in Africa, without a doubt, but this appointment tarnishes the good work this organization has undertaken in recent years," spokesman Rick Roth told The Canadian Press in an email Friday.
"We believe that African nations by-and-large are above the sort of maniacal nature of this brutal dictator, who has long suppressed the freedoms and dignity of his own people."
Roth said Canada maintains an "unwavering" commitment to Africa, "but we fundamentally believe that African nations should reject the type of corrupt conduct Mr. Mugabe has shown, including his blatant disregard for human rights."
The International Crisis Group also condemned Friday's appointment.
"Frankly I don't believe the elevation (Mugabe's appointment) is anything than symbolic," said Piers Pigou, Southern Africa project director for the International Crisis Group. "His elevation sends a negative signal of African solidarity with leaders who've misruled their countries."
Traditionally, the AU chairmanship is given to the leader of the country hosting the next summit, but exceptions have been made as in 2005 when it was the turn of Sudan's Omar al-Bashir.
African leaders bowed to international pressures in the uproar over killings in Darfur, passing over al-Bashir and instead kept Nigeria's Olusegun Obasanjo for a second year.
"During my tenure as chair, I will deliberately provoke your thoughts to pay special attention to issues of infrastructure, value addition, agriculture and climate change," Mugabe told African leaders.
— with files from The Associated Press