Speaking at the party's annual general meeting in Halifax on Sunday, the Opposition leader told supporters that since taking the reins in October, the Liberals have done nothing to address the province's debt.
He urged the government to balance the books ahead of the spring budget in order to provide tax relief and create jobs.
"That's where the rubber is going to hit the road on whether there is any substance to the Liberal government or not," Baillie said during a question and answer period with party members.
"You will see a very vigorous and loud official Opposition if they break their promises to the people of Nova Scotia or leave our problems unaddressed."
The province's finance minister kicked off consultations Friday in preparation for its spring budget.
Diana Whalen has already vowed to reduce departmental spending by one per cent, except for Health and Education.
Last month, Nova Scotia's then-auditor general said the province's net debt stood at $13.9 billion at the end of the 2012-13 fiscal year, a 5.3 per cent increase over the previous year.
Baillie also spoke about the lessons learned from October's election and vowed to run a tougher campaign against the governing Liberals during the next election.
Baillie said campaign teams in each riding will have to reorganize and work together to be successful, adding that some Tories who sought nominations in the last election but did not win were causing issues within the party.
"When we nominate our next slate of candidates, let's all agree right now, whether your on the winning side of that PC nomination or not, we are going to unite behind the PC candidate," he said.
On Saturday, Progressive Conservative members overwhelmingly endorsed Baillie as their leader in a review vote.