Sources say Natural Resources Minister Joe Oliver will step into the job, which Flaherty held for eight years.
The Prime Minister's Office hasn't confirmed Oliver's promotion but the 73-year-old cancelled an event he had scheduled in British Columbia for today.
A swearing-in ceremony is expected to take place at Rideau Hall, where Oliver has been spotted this morning, along with Ontario MPs Ed Holder and Greg Rickford.
That has fuelled speculation that Rickford — currently the junior minister for science and technology with reponsibility for the Ring of Fire mineral deposit in northern Ontario — will take over from Oliver at Natural Resources, and be replaced in his current portfolio by Holder.
Oliver is widely respected within the Harper cabinet and among Conservatives, and is known on Bay Street, having spent decades working there as an investment banker.
Oliver has a much different demeanour than Flaherty though, and is often gruff and highly partisan.
Flaherty has been the only finance minister the Harper Conservatives have had since they came to power eight years ago.
He managed the country's economy through one of its worst economic crises in 2008-2009, running up large deficits but leaving the books virtually balanced after his Feb. 11 budget.
Flaherty has said his decision to resign was made with his family earlier this year and is unrelated to a rare skin condition that requires him to take medication that led to weight gain and apparent fatigue.
"As I begin another chapter in my life, I leave feeling fulfilled with what we have accomplished as a government and a country during one of the most challenging economic periods in our country's history," Flaherty said in a statement issued late Tuesday afternoon.
An official in the Prime Minister's Office said Flaherty would be retaining — at least for now — his seat in the House of Commons, meaning he remains an MP for the time being.
Observers have said whoever takes over from Flaherty will have their work cut out for them.
Also on HuffPost