Quebec provincial police said the incident happened on Friday at around 4:45 p.m. ET on a private lake just off Hébert road, in Ascot Corner.
The two were skating on thin ice when it collapsed, and the boy fell into the freezing water. The man tried to rescue him, but slipped into the lake, which is about two metres deep.
Other members of the family saw the incident from an adjacent house and tried to save the pair.
The grandfather was pulled from the water by emergency workers, but died soon afterwards.
The boy was rushed to hospital in critical condition and died Saturday morning.
His grandmother is also in hospital, suffering from hypothermia. She is expected to recover.
Gerard Payeur, 64, and Nicolas Payeur, 9, were both from Ascot Corner.