Those levels of government need to get their finances in order, the PBO said, suggesting they could try to find savings in health care or other programs, receive greater transfer payments from Ottawa or come up with new ways to raise revenues.
"Canada's subnational governments (provincial, territorial, local, and aboriginal) cannot meet the challenges of aging demographics," says the report.
"They must make significant policy changes to put public debt on a sustainable path."
Despite the dire warning, the PBO says it's not too late for provinces, territories and municipalities to get their public debt under control.
"The fiscal gap of subnational governments is large, but not insurmountable," says the report.
"If provinces are able to control excess cost growth, an increase in the (Canada Health Transfer) escalator to account for population aging when it is reviewed in 2024 would eliminate much of the fiscal gap of subnational governments while still leaving the federal government with considerable fiscal room.
"The remaining subnational fiscal gap could be closed with a mix of modest spending restraint and tax increases."
The PBO was more optimistic about the federal government's ability to deal with Canada's aging population, saying Ottawa has enough fiscal wiggle room to deal with the challenges of people getting older.
The federal government could boost its spending or cut taxes while maintaining sustainable public debt, added the PBO.
The report also says the Canada Pension Plan and Quebec Pension Plan are both able to pay for the projected increase in beneficiaries while remaining sustainable.
Also on HuffPost: