Pictures captured by a remote underwater vehicle operated by the Navy confirm there are no bodies inside the Miss Ally. The images show the wheel house and sleeping quarters are not attached to the hull.
The Department of National Defence and the Coast Guard are returning to their normal operations and the mobile command centre is packing up, say police.
A private dive team searched the wreck on Saturday and came up empty-handed.
HMCS Glace Bay arrived at the scene of the overturned Miss Ally overnight.
A plane spotted the hull just before 10 a.m. on Saturday morning.
The 13.5-metre vessel, which was on an extended halibut fishing trip, was last spotted by the Coast Guard on Tuesday, floating upside down more than 100 kilometres offshore.
Kenny Hatfield, the uncle of one of the fishermen lost aboard the Miss Ally, said he was hopeful the families would finally get some closure.
"I had a little bit of hope yesterday morning when I heard on the news that they found the boat. I said, 'Good. They'll go out, take them off the boat.' That was my last thought, they would have been still aboard the boat," he told CBC News.
"Then I heard on the news there was nobody in the boat. My hopes just dropped like that."
Sunday church services packed
The formal search for crew members Billy Jack Hatfield, Joel Hopkins, Katlin Nickerson, Steven Cole Nickerson and Tyson Townsend was called off Tuesday night after officials concluded there was little hope any of the men would have survived the rough seas and cold water.
A private fishing boat continued the search.
A joint operation involving Department of National Defence and Canadian Coast Guard crews had launched an extensive search for the vessel on Thursday morning after families and friends pleaded with authorities to keep looking.
Sandy Stoddard, a fisherman from Woods Harbour, says he hopes some lessons can learned from the tragedy.
"I would only pray that the government would look at this here incident and say, 'We have to do a better job and not let this happen to somebody else.' Because you want to remember that these fishermen in this community, they don't know protocol. They probably don't know what the word means and I don't want you to think for one minute that they're uneducated people, because they wouldn't want to know what protocol meant. Because we are doers. And when something needs to be done, we go do it," he said.
A Baptist Church in Woods Harbour was overflowing with congregation members on Sunday morning, the CBC's Michael Dick reported.
Five candles were lit for the five missing men.
Most of them had young families.