Tom McGinn, executive director of the New Brunswick Road Builders and Heavy Construction Association, said Tuesday that cuts to the capital budget in recent years have meant lost jobs and more roads that are in disrepair.
"Our twinned highways are OK, but it's our rural roads that seem to be neglected," he said. "Those are the ones that we feel should be getting more attention."
McGinn says there were about 7,000 people working directly and indirectly on highway and road infrastructure three years ago, but that has been reduced by about 1,000 because of the cuts.
The capital budget for the Department of Transportation was $466.3 million in 2010-11 but fell to $269.7 million in 2012-13.
Williams said he'd like to spend more money on roads, but finances are tight and he's competing for funding with other departments, such as health and education.
"The situation is that here in New Brunswick we have got to live within our means," Williams said Tuesday.
"Certainly I would like to invest more on roads and highways in New Brunswick but the reality is that there are only so many dollars to go around."
Auditor general Kim MacPherson highlighted cuts to the road budget in her report last December, saying delays to the work could cost five or six times as much to fix in the future.
McGinn said there are a number of arguments in favour of fixing roads now.
"They can fix our roads, save future taxpayers money, create jobs and give our economy a boost," he added.
"After the underfunding of the last three years we don't think we can wait any longer."
Williams said his department is doing a good job of maintaining the 18,749 kilometres of roads and 2,600 bridges in the province.
All road infrastructure is regularly inspected to ensure it is safe, he said.
"Never would we keep an infrastructure knowing that it posed a risk for motorists," Williams said.
Williams said he would be making a pitch for more funding at a caucus retreat this weekend.
The province's capital budget for the coming year is expected in December.