Flaherty resigned in a surprise move last month following an eight-year tenure overseeing the country's treasury. He said his decision was not due to health reasons but because he planned to pave the way for a re-entry to the private sector.
The cause of his sudden death at the age of 64 has not yet determined, but sources told CBC News he had suffered an apparent heart attack on Thursday.
It was also known that the Ontario MP for Whitby-Oshawa had a rare and painful skin condition called bullous pemphigoid, which required steroid treatments.
Together with his wife, Ontario Tory MPP Christine Elliott, Flaherty raised three triplet boys — John, Galen and Quinn.
Flaherty, who grew up in an Irish household and had a penchant for statement-making green neckties, was born Dec. 30, 1949 in Lachine, Que., an area of Montreal's West Island.
Princeton hockey scholarship
He earned a scholarship to play hockey at Princeton University and also graduated with a law degree from Osgoode Hall at York University in Toronto, practising law for 20 years before entering politics.
He first ran in Ontario in 1990 but lost, coming in third place in the riding of Durham Centre.
Five years later, he won a seat in the province's legislature as a rookie member for Whitby-Ajax. In 1997, he was tapped to join Mike Harris's cabinet as minister of labour, and also went on to serve as attorney general.
Flaherty took on Ontario's treasury file as finance minister in 2001 until 2002, when he launched an unsuccessful bid to succeed Harris as leader of the Ontario PC Party, losing to Ernie Eves.
He entered federal politics when he was first elected to the House of Commons in 2006.
In that first year, he announced the government would impose new taxes on income trusts, breaking a Conservative campaign promise but defending it as a measured response. The controversial move eliminated over $20 billion in stock market value overnight. He also dropped the GST to six per cent from seven per cent in his first Conservative budget, and eventually to five per cent in 2008.
Flaherty won wide recognition for steering Canada through the 2008 financial crisis and global recession and buoying Canada's economic reputation. In conjunction with the Bank of Canada, he slashed interest rates in a bid to spur businesses and households to borrow money and spend.
He also introduced tax-free savings accounts in 2008 to allow people to earn tax-free investment income, and tightened mortgage lending rules several times to slow the apparent runaway growth of mortgage debt.
A man of his convictions
Until his resignation, which was announced after the markets closed on March 18, he had been the only finance minister for the Harper Conservatives since they took power in 2006.
Friends remembered Flaherty as a man of his convictions. He was deeply principled until the end, suggesting in February he no longer supported bringing in income splitting for couples with children — a policy that other Conservative MPs were backing in line with a Conservative promise.
Colleagues on Parliament Hill from all political stripes expressed shock at the former minister's sudden death.
Next to Prime Minister Stephen Harper, Flaherty was the most familiar face of the government, commanding respect from colleagues, even when they didn't see eye to eye politically.
NDP Leader Tom Mulcair described Flaherty as a friend and said the House of Commons would feel a "profound loss" in his absence.
"We're very very, sad for the loss of a great Canadian, Jim Flaherty, who was an extraordinarily dedicated public servant, and he'll be greatly missed by all of us," Mulcair told reporters.
Former Liberal interim leader Bob Rae, who crossed swords with Flaherty many times during question period in the Ontario Legislature and on Parliament Hill, said although Flaherty was deeply partisan, he made friends across the aisle.
Champion for disability issues
"I worked with his wife, Christine, in charitable work in Toronto and got to know Jim very well. As things passed, we got to know each other socially a bit," he said, adding that "Jim sent me the kindest note when I left politics."
Although Flaherty's legacy may be measured by his effectiveness at keeping the books balanced during one of history's most calamitous economic periods, he was also passionate about improving quality of life for disabled Canadians.
In 2007, Flaherty introduced the Registered Disability Savings Plan, which is meant to help parents ensure disabled children have long-term financial security.
In a tweet, elite wheelchair athlete Jeff Adams called Flaherty a friend and noted that he was a "strong supporter" of Variety Village, an all-abilities sports and fitness facility.
At a routine speech about the disability assistance plan in 2011, the finance minister was visibly moved, fighting back tears as he discussed the plan and the challenges that parents of disabled children face.
One of Flaherty's sons, John, has a mental disability and shared an interest in baseball with his father.
The former minister was also a father figure of sorts for younger Conservative MPs. Kellie Leitch considered him to be a mentor, as did James Rajotte, chair of the House standing committee on finance.
In a statement, Leitch credited Flaherty with getting her into politics, describing him as her "champion."
"Canada has lost a giant and our our government has lost one of its most respected and capable members," she said.