Msgr. Luc Cyr, the archbishop of Sherbrooke, Que., began the ceremony at Ste-Agnès Church by welcoming the friends and families of the victims, as well as the many dignitaries that joined the ceremony.
Prime Minister Stephen Harper, Gov. Gen. David Johnston, and Premier Pauline Marois were among the dignitaries attending the service.
In a touching moment, firefighters received loud applause from the crowd as they lined up across from the church.
The service opened with a reading of the names of all 47 victims, followed by a meaningful excerpt from Isaiah 49:16: "I will not forget you, I have your names etched upon the palms of my hands.”
'Unheard of tragedy'
In his message Father Steve Lemay, the parish priest, described the town of Lac-Mégantic as a place normally filled with life, brotherhood and fraternity.
“What happened here in our town, in Lac-Mégantic?” he asked the crowd. ”An unheard of tragedy that brought us incomparable suffering.”
Father Lemay described the victims as individuals who were full of vitality.
“Our town is marching, mourning and it is accompanied by a large crowd,” he said.
Stephen Harper: Canada stands in solidarity
After the service came to an end, Prime Minister Stephen Harper briefly addressed media.
"This has been a very emotional day," he said. "It's still very difficult to absorb this when you see all of these families that have been affected," he said.
Harper added that he was also there to express "the solidarity of all Canadians with the people of Lac-Mégantic."
Father of victim says he depends on faith, family
Some 700 places in the 1,000-seat church were reserved for loved ones of the deceased, with the remaining pew spots set aside for locals, volunteers and dignitaries.
The father of one of the victims spoke to CBC News this morning before the mass began.
His daughter, Kathy Clusiault, 24, was killed in the blast three weeks ago.
She lived above Musicafé, a popular bar near the train tracks, where many of the victims were enjoying the summer evening when the train careened of the tracks.
Jean Clusiault said his faith and his family's support has helped him through his grief.
"I’ve got great support and I know the preacher who is here. He’s a friend of mine and he helped me a lot," he said.
Forty-seven people were killed when the train carrying crude oil derailed, causing multiple explosions at the centre of the bustling town.
Forty-two victims have so far been found in the city's devastated core. Crews are still searching for the remains of five missing people, who are presumed to be dead.
The tragedy has triggered several lawsuits, a police criminal investigation and a probe by federal transportation-safety officials.
Quebec and the federal government have each promised $60 million for emergency assistance and longer-term reconstruction help for the town.
Ottawa has also revamped some of its rules on train transport, following the advice of the federal Transportation Safety Board.