Until mid-March, the jocular, wise-cracking Flaherty had been the only finance minister the Harper government had ever known.
He died suddenly Thursday at 64.
His death left MPs on all sides of the political spectrum shocked and tearful and prompted an outpouring of condolences.
The United States' new ambassador to Canada said Flaherty was well-liked on both sides of the border.
"His leadership and integrity earned him the respect and admiration of Canadians and Americans," Bruce Heyman said in a statement.
"His public service, to Ontario and to all of Canada, made an enormous positive difference in the lives of people. His spirit, undaunted good cheer, and great love of life are an example to all of us."
Flaherty took the Finance portfolio when Prime Minister Stephen Harper formed his first government and held it through four elections, cabinet changes, world economic shocks and a bout with a rare skin disease that put him on steroids.
The condition, however, was not thought to be life-threatening and the cause of death was not immediately announced.
His death came as a shock. A shaken Harper called him a colleague and partner. Some of his fellow Tory MPs wept at the news. Political foes offered praise.
NDP Leader Tom Mulcair, who was prepared to savage Conservatives for perceived flaws in budget after budget, squeezed his eyes against tears as he recalled Flaherty's "pleasant demeanour and strength of character."
Flaherty was a stocky leprechaun with a twinkling eye in a wide Irish face. He had a seemingly bottomless wardrobe of green neck ties, a crooked grin and a quirky sense of humour that took the edge off his parliamentary barbs.
After his maiden speech in the House of Commons, he was replying to questions when the Speaker urged him to keep his answers short.
"I assure you, Mr. Speaker, that I am always short, but I will also try to be brief," he replied to chuckles from all around.
James Michael Flaherty, Jim to all, was born in Lachine, Que., on Dec. 30, 1949.
After attending Bishop Whelan High School and Loyola High School in Montreal he earned a BA at Princeton University and a law degree from York University's Osgoode Hall Law School.
He was a founding partner of the law firm Flaherty Dow Elliott before entering politics in 1990, when he ran and lost provincially in the riding of Durham Centre.
He ran again in 1995, and won a seat in the legislature.
The new member for Whitby-Ajax entered Mike Harris's Tory cabinet in 1997 as labour minister. He also served as finance minister, attorney general, enterprise minister and deputy premier under Harris and his successor, Ernie Eves.
Flaherty twice sought the Ontario Conservative leadership unsuccessfully.
He sought to succeed Harris in 2002, but lost to Eves, his predecessor as finance minister.
In his leadership campaign, Flaherty proposed a tax credit for parents to send their children to private schools and said he would privatize the Liquor Control Board of Ontario.
He made some barbed comments about Eves during the leadership campaign and found himself with the low-key job of minister of enterprise in the next cabinet.
After the Conservatives lost the 2003 provincial election, Eves said he would step down the next year. Flaherty ran again for leader, but lost to John Tory.
In 2006, Flaherty jumped to federal politics. He won in Whitby-Oshawa as the Tories former their first minority government.
As Finance minister, Flaherty never saw a tax that couldn't be cut down a bit. In 2006 he chopped a percentage point to drop the GST to six per cent. He shaved off another percentage point in 2008.
In 2006, he also announced changes to the tax rules governing income trusts which sparked anger and prompted special hearings by the Commons finance committee.
Flaherty's budgets kept coming up with ways for people to reduce their taxes.
In 2007, he introduced the Registered Disability Savings Plan to help Canadians with disabilities and their families save. One of Flaherty's triplet sons has a mental disability and the minister wept openly in announcing the plan.
The next year he brought in the Tax-Free Savings Account, which allows people to earn tax-free investment income.
There were also targeted tax cuts to help everyone from truckers to people buying sports equipment. He raised the basic personal deduction to get more low-income people off the tax rolls entirely, cut a point from the lowest personal tax rate and raised the upper limits of the two lowest tax brackets.
In his 2013 budget, he boasted that the average family of four was saving $3,220 in taxes compared with their bill in 2006.
While he was a strong fiscal conservative, Flaherty wasn't afraid to plunge into deficit to stave off the effects of the 2009 recessions.
He took the balanced budgets of the Liberal years deep into the red as he pumped money to support infrastructure projects and keep the economy from stalling.
He pledged, however, to get the books in balance in time for the federal election expected in 2015. He was within an eyelash of doing that with his last budget in February.
He is survived by his wife, Christine Elliott, herself a lawyer and a politician. When he gave up his provincial seat to run federally, she won it, leaving them representing roughly the same region in Ottawa and Toronto..
The couple had triplet sons, born in 1991, Quinn, Galen and John.