That changed Tuesday when 22-year-old Derek Drouin finished in a three-way tie for third in the men's high jump to win bronze at London's Olympic Stadium.
Canada's Duncan McNaughton won gold in high jump at the 1932 Games in Los Angeles before Joy won silver on home soil 44 years later.
"Now we have a gold, silver and bronze for men," Joy said from Ottawa, where he left work to watch the final in an adjacent building. "We've had so many great high jumpers. We just haven't been able to get a medal since mine."
Drouin's bronze gave Canada its 11th medal at the Games to remain 12th in the overall standings.
"Greg Joy," Drouin said. "He's obviously an incredible jumper, I feel like I'm in good company with him, and to be the first one to do it since a long time before I was born, it feels pretty great."
The 56-year-old Joy says Canadian high jumpers have been close on the Olympic stage since his second-place showing in Montreal.
"It's so technical that anything can go wrong and if a little thing goes wrong, you're out," said Joy. "The bar has its own personality sometimes. You make a little mistake on the wrong day, and it's only every four years. It's tough. That's been shown over and over again."
Drouin, from Corunna, Ont., and the other two jumpers he will share bronze with cleared 2.29 metres without any misses. Each then failed in three attempts at 2.33 metres.
Although Drouin's medal came as a surprise, it was an otherwise disappointing day for Canada.
Victoria triathlete Simon Whitfield — a two-time Olympic medallist — was taking part in what is likely his last Games on Tuesday when the unthinkable happened.
The 37-year-old had just gotten out of the water and was barely on his bike when he was involved in a crash that forced him out of the race.
"Not how I envisioned this chapter ending," Whitfield posted on his Twitter account of the disappointment. "It's been a remarkable Olympic journey, more then I could have ever imagined."
Track cyclist Tara Whitten also had a disappointing day and was in tears at the end of the women's omnium after a fourth-place finish.
"We all just want to do it so badly for Canada," said the 32-year-old from Edmonton. "I just gave it everything I had today and it just wasn't enough."