During his annual address to the Halifax Chamber of Commerce, McNeil said for too long governments have created programs but haven't ensured the public is getting value for what it is paying for.
"We shouldn't be doing all the things that we are doing," said McNeil.
"It's absolutely critical that we as a government are getting the objectives out of the money we are spending on behalf of all Nova Scotians."
Government ministers and departments have also been instructed to review programs in order to determine which are necessary, he said.
The government recently received a report recommending major changes to some taxes, including the introduction of a carbon tax and a broadening of the harmonized sales tax to cover items previously exempt.
That report compiled by former Ontario cabinet minister Laurel Broten also called for lower income and corporate tax rates.
McNeil was asked after his speech whether he would consider some of the report's recommendations.
"Tax changes are difficult and I can't afford the revenue loss to be perfectly honest," he replied.
He later told reporters that he wasn't saying there wouldn't be any changes to taxes, but that there would be no major moves because "we need the revenue we have."
Pollster Don Mills said he believes McNeil is laying the ground for a difficult budget.
"I think we are going to see the end of some programs and maybe some programs that are surprising," said Mills.
McNeil also said he will hire a deputy minister from the private sector in the coming weeks whose sole purpose will be regulatory review and reform.
He said he also hopes to work with New Brunswick Premier Brian Gallant to create a joint regulatory policy that would eliminate red tape and streamline rules for the business sector.
Valerie Payn, the president of the Halifax Chamber of Commerce, said businesses understand significant tax cuts are difficult as the government looks to balance its books in the near future.
But she applauded the premier's promise to streamline regulations, saying the initiative can help trim business costs.
"It's hugely damaging if you have a regulatory system that's onerous and hard to navigate," said Payn.