The Quebec Police Ethics Committee said Vincent Langlais displayed gross incompetence and an obvious lack of judgement.
In 2009, Yvon Trudel's nursing home in Salaberry-de-Valleyfield called police to report him missing.
Langlais found Trudel at Plaza Deli, a restaurant across the street from the residence.
Waitress Rachel Carrier was working and said she saw the officer grab Trudel and push him to the ground.
"He seemed to be so affected, the poor man. I could not stop crying. I was really mad," she said.
Trudel suffered a heart attack ten days after the incident. He died four months later.
The committee decided that Langlais did not have the right to forcibly drag Trudel out of the restaurant.
Langlais has 30 days to appeal the ruling.