Robert Ghiz, the premier of Prince Edward Island, said he would have welcomed the prime minister's ideas at the table on how to improve the country's "sluggish" economic growth.
"There is a difference when he is here," Ghiz said late Thursday after arriving in the port city.
"Whether it's skills development, university funding ... immigration, trade, all these things. There are federal ties in all those areas."
Nova Scotia Premier Darrell Dexter, who is hosting the meeting, said it's challenging for provinces to devise their spending plans without the prime minister's input.
"It's difficult for us to know how we can best blend our own planning in with the funding of the federal government if we don't know what their plans are," Dexter said.
The idea for such a gathering came up during the last premiers' meeting in July, when they agreed to discuss how they could protect their provinces from a global economy in flux. That was before revised figures showing a higher federal deficit and growing concerns over the possibility of the U.S. economy going over its so-called fiscal cliff at the end of next month.
The premiers invited Harper but he declined. His office has said he has met regularly with the premiers and the economy remains his prime concern.
Harper's absence from the meeting has also become fodder for opposition parties in the House of Commons this week, where they accused him of shirking his responsibilities.
"Mr. Speaker, precisely what Canadians want from their leaders is not more meetings," Harper said in response Thursday. "They want action from this particular government."
Still, that kind of response has not sat well with some, including Ontario Premier Dalton McGuinty.
"I think we're all disappointed that we couldn't get an opportunity to meet with the prime minister and to build a strong plan for economic growth together," McGuinty said earlier this week.
"We'll see what we can do by way of a concerted effort."
Quebec Premier Pauline Marois also criticized Harper for not attending, saying she couldn't understand why he wouldn't attend given his interest on the economy.
Late Thursday, Marois and Alberta Premier Alison Redford had a separate meeting to discuss the possibility of shipping oil from the West. They agreed to explore the economic opportunities and possible environmental effects of the project.
Glen Hodgson, chief economist at the Conference Board of Canada, said the premiers should avoid the divisions that have defined some of their past meetings and attempt to make progress on matters under their control, such as boosting immigration and knocking down interprovincial trade barriers.
"What are provincial governments doing to deepen the skills of their workforce or to encourage more immigrants to work into the Canadian workplace and get rid of barriers between provinces?" he said.
One highlight of the meeting is a presentation by Bank of Canada governor Mark Carney, who will discuss the bank's outlook for the global and Canadian economies as well as national inflation.
Carney has said the U.S. fiscal cliff is the most imminent threat facing the Canadian economy, and he has also repeatedly warned that personal debt levels have reached record highs, posing a risk to the economy if consumers can't afford to carry their debt when interest rates rise.
Manitoba Premier Greg Selinger said he'll be listening carefully to any guidance Carney can provide as he reviews his province's finances.
"We'll be looking to him for what his specific ideas are," Selinger said.
The meeting comes a week after federal Finance Minister Jim Flaherty released the government's revised deficit projections that show a deficit of $26 billion — up $5 billion from the March budget forecast — as a result of global economic weakness carving into commodity prices and tax revenues.
The gloomy numbers have also prompted Ottawa to delay its hopes of balancing the books until 2016-2017, a year later than previously forecast. Despite that, Flaherty said he would not touch fiscal transfers to the provinces.
Dexter said one issue he would like to address is the $8.8-billion Building Canada Fund. The federal infrastructure program expires in 2014.
"We're given to believe there will be a new infrastructure program, but we're not told what it is," he said.
In an email, Harper spokesman Andrew MacDougall said Infrastructure Minister Denis Lebel has been going through a six-month, national consultation on the fund.