It's unclear whether the Grace Foundation asked the five directors to leave or they chose to leave.
The shuffle comes on the heels of the Saint John-based foundation quietly dropping its request for a refund.
The Grace Foundation had paid Trudeau to speak at a fundraiser for the Church of St. John & St. Stephen nursing home last summer, when he was still an MP but before he'd become party leader.
The event ended up losing money, however, and in March, a board member wrote to Trudeau, asking for a refund.
Trudeau had offered to compensate any charities that had paid him to speak at events and was "waiting to hear back from [the Grace Foundation] as to how exactly they wanted to settle this matter," said press secretary Kate Monfette.
The foundation's chairman Ian Webster recently spoke to Trudeau's chief of staff, she said.
"He said they would not seek reimbursement, and they consider this matter completely closed," Monfette told CBC News.
"It's not to us to comment on their decision," she added.
Board 'deeply distressed'
The decision comes just weeks after Webster had issued a statement, saying the foundation had never intended for its dispute with Trudeau to become a political matter, and the board was "deeply distressed about many statements made from various persons."
Five of the 10 board members are no longer with the foundation, according to the organization's website.
Susan Buck, who had written the letter requesting the refund because the event was a "huge disappointment and financial loss," is no longer listed as a board member.
Judith Baxter, who gave Buck's letter to Conservative MP for Fundy-Royal Rob Moore, is also among those no longer listed.
Baxter, who was appointed by the Harper government to the Canadian Museum of Civilization's board in 2007 and again in 2011 and whose husband, Glen, is on the executive of Moore's riding association, declined to comment on Tuesday.
The foundation's recent statement said the board did not authorize anyone to approach Moore or any political person on the speaking fee dispute.
But Moore has maintained the charity asked him to pressure Trudeau to return his fee after a letter to Trudeau didn't work.
Moore was appointed minister of state for the Atlantic Canada Opportunities Agency and regional minister for New Brunswick and Newfoundland and Labrador during Harper's cabinet shuffle last week.
The federal Conservatives have criticized Trudeau for taking money from charities, accusing him of caring more about making money than serving the public.
Also on HuffPost