Percy Davis was holding a butcher knife when he was shot twice in the parking lot of a mall in northeast Edmonton in 2008.
The Alberta Serious Incident Response Team ruled the officer was justified in firing his gun.
Davis was convicted of assaulting a police officer, assault with a weapon and possession of a dangerous weapon.
He received a suspended sentence and one year of probation.
The high court recently ruled the trial judge made an error by ruling Davis had the burden of proving that the officer had used excessive force.
Davis had argued his Charter of Rights were violated.
"The appeal is allowed and the appellant’s conviction is quashed," wrote the court.
"As the parties have requested in such an event, a new trial is ordered on the same charges."
Court previously heard Davis, then 26, was holding the knife while riding his bike in the parking lot, when a police car showed up. He charged at the open window on the driver's door with his knife raised.
The officer inside ducked, got out of the car and demanded Davis drop the knife.
Davis walked away and headed towards the mall. The officer followed him and decided to shoot when it became clear that Davis would not drop the knife.
Davis was hit in the neck and right chest and reportedly lost the use of a lung and an arm.
The man's father, an evangelical pastor, said at the time that his son was a loner who had grappled with anger issues. He said the officer should have been carrying a Taser to subdue his son.
Also on HuffPost