CBC News obtained documents that show Penashue exceeded the spending limit in the 2011 election campaign by almost $4,000, and that he had obtained a $25,000 loan from the Innu Development Limited Partnership — a firm managed at the time by Penashue's brother–in–law — after some cheques written by his own campaign bounced.
The documents also reveal irregularities in Penashue's campaign in the riding of Labrador, including cheques with no names, handwritten receipts and a donation with no name attached.
Harris, who represents St. John's East in Parliament, said he does not accept the explanation from Penashue's office, which pushed the blame to campaign manager Reg Bowers, who oversaw the books.
"Ultimately the buck stops with the candidate, so Mr. Penashue has to take responsibility," Harris said.
Penashue, who defeated Liberal incumbent Todd Russell in last year's election, is the minister of intergovernmental affairs, as well as Newfoundland and Labrador's federal cabinet representative.
Elections Canada has given Penashue time to explain the campaign spending. A statement last Friday from Penashue's office said it has "been in contact with Elections Canada to work on making needed amendments."
In an interview with CBC News, Harris said sticking to a spending limit is one of the basic rules of a campaign.
"How much money are you spending? You're not supposed to spend over a limit — that's the first thing that you're told," he said.
"You're only allowed to spend a certain amount of money."
While Penashue's office has said that Bowers, as the official agent for Penashue, made "some errors" during the campaign, those mistakes did not prevent the federal government from appointing him to the Canada–Newfoundland and Labrador Offshore Petroleum Board in December.