The gold went to the three-time world champion Germans while Britain took the bronze.
It's Canada's fifth medal at the London Games.
The Canadian men entered the race as defending Olympic champions but only have three returning members of that 2008 crew — Toronto's Andrew Byrnes, Victoria's Malcolm Howard and coxswain Brian Price of Belleville, Ont. — in the London boat.
Canada started sluggishly at the regatta, finishing last in its heat. But the Canadians bounced back to place second in their repechage and advance to the final at Eton Dorney.
The women's eight goes for gold Thursday.