A former journalist, Graham was appointed to the Senate in 1972 by Pierre Trudeau and served until his mandatory retirement in 2004 at age 75.
He earned several other positions in the course of his 46-year-long political career, including president of the Liberal Party of Canada. He also served as deputy leader, and then leader of the government in the Senate.
Graham also spent two years representing Nova Scotia in Jean Chretien's cabinet.
Premier Stephen McNeil extended his condolences to Graham's family and friends, calling Graham an example everyone should aspire to.
"Al Graham's contributions to the political landscape in Canada are well known," he said in a statement Thursday. "He was passionate about our region and its people and he represented us with great distinction."