Oliver lost his Toronto–area seat in a hard-fought campaign to Liberal Marco Mendicino, the lawyer who won the nomination over Eve Adams, a controversial former Conservative who crossed the floor to join the Liberals earlier this year.
"We were going to keep taxes low and balance the budget, and the other guys were going to raise taxes and create massive deficits," said outgoing Finance Minister Joe Oliver during an interview on CBC News Network's Power & Politics on Thursday.
"You'd think that would be a winning strategy but it wasn't enough to overcome this momentum," said Oliver of the Liberal surge that swept the country in Monday's election.
Oliver, who was elected to the House of Commons in 2011, conceded there were "some issues" with the Conservative campaign.
"We had a rock-solid base but we couldn't expand it," he told host Rosemary Barton.
"When people in the party analyze this, as I know people in the party are in the process of doing, we have to think about what in terms of policy and communication we can do to get that extra seven or 10 per cent."
'Focus on a broader appeal'
Except for a brief trip to Turkey for the G20 finance ministers and central bank governors meeting halfway through the campaign, Oliver was rarely heard from during the 78-day campaign.
Oliver rejected any suggestion that he was avoiding the media, saying he spent the majority of his time going door-to-door in Eglinton–Lawrence talking to voters about "pocketbook issues" in an effort to secure his reelection bid.
Though Oliver was responsible for tabling the final budget on which Conservatives ran, it was Conservative Leader Stephen Harper who made the pitch to voters nationally.
By the end of the longest election campaign in modern Canadian history, Harper was using props and the "ka-ching" sound of a cash register to illustrate the impact of "Liberal tax hikes."
Asked if he was muzzled, Oliver let out a brief laugh. "Let's just say, I felt that I could do more to help the party by being out there."
"We had a very strong team and it's a team that was supportive of the prime minister and perhaps showcasing a bit more would have helped...it would have shown here's a strong team that can articulate the Conservative message."
Oliver said he was certain the party, though now relegated to Opposition status, would eventually recover from Monday's loss.
"While it was a significant defeat, it wasn't devastating," he said.
"We have to focus on a broader appeal, I think, without betraying our core principles. There's no reason in the world why we can't do that."
Oliver had big shoes to fill when he took over the finance portfolio in 2014 from the late Jim Flaherty — Harper's only other finance minister since the party came into power in 2006.
Until then, Oliver had served as minister of natural resources, a controversial portfolio for a government unapologetic for its support of the oil sands.
Oliver made headline news in 2012 when he blamed environmental activists and other "radical groups" for threatening to block energy projects by using funding from special interest groups to undermine Canada's national interest.
That same year, he insisted that sweeping changes to the environmental review process would not compromise environmental protection.
In 2014, the government approved the Northern Gateway pipeline with some 200 conditions, but aboriginal leaders said no pipeline projects would be build without their approval.
After nearly a decade in power, the Conservatives were not able to get any oil pipelines built.
Asked about his political legacy, Oliver said he was proud of the balance budget he tabled earlier this year which included funding for a new centre for aging and brain health innovation in his riding.
"This is going to be world class organization... it's going to be Eglinton–Lawrence's gift to Canada and Canada's gift to the world."
Oliver, 75, was optimistic about the future.
"I have a fair amount of private and public experience. I have a lot of energy still and we'll see where it goes," he said.
"I don't intend to be feeding the pigeons."
Also On HuffPost: