Sheridan tabled a $1.6 billion fiscal blueprint that largely holds the line on spending as it tries to pull itself out of choppy fiscal waters.
But tax hikes will soon hit hard the wallets of the province's 146,000 residents.
The HST will kick in on Monday, a change that will see gas prices rise by about eight cents a litre and smokers paying at least an extra dollar for every carton of 200 cigarettes.
Prince Edward Islanders now pay a combined provincial sales tax and goods and services tax for a total of 15.5 per cent. The new HST will be 14 per cent but will apply to more items such as adult clothes and electricity bills.
The small business income tax rate will also rise to 4.5 per cent from one per cent, an increase that is expected to generate another $6.6 million per year.
Sheridan said the new tax regime will help the province steer clear from running continuous deficits.
"We need our deficit to structurally change," Sheridan told a news conference prior to delivering the budget.
"We have to change the way that we do business, and this is it."
Sheridan is projecting a deficit of $58.9 million for 2013-14, down $10.3 million from this fiscal year but $25 million more than what he had forecast last year.
The province is on track for a surplus in 2015-16 — a year later than expected — but the public was clearly against aggressive cuts, he said.
"The majority of those consulted said we should push our balanced budget target out one year," Sheridan said. "This will allow the global economic recovery to take effect, which will result in increased revenues for the province."
Progressive Conservative finance critic James Aylward said the budget unfairly targets the small business sector.
"The majority of businesses were looking forward to getting rebates to be able to hopefully hire more people and grow their businesses," Aylward said.
"But if you're jacking up the taxes all of a sudden by 350 per cent ... it's pretty easy to figure out that I'm not going to be able to grow my business."
The net debt will reach about $2.1 billion by March 31, 2014, or close to $14,400 for every man, woman and child in P.E.I.
"At that point we would reach the pinnacle of net debt and then begin the downward trend," Sheridan said.
Some departments are getting small funding increases, including $16 million for health and $4.4 million more for the Department of Community Services and Seniors.
The provincial government received $25 million from the federal government this fiscal year to assist with the transition to the HST. It will get a final payment of $14 million in 2013-14.
Also on HuffPost