The vessel was five nautical miles southwest of Burgeo doing routine work on buoys when it ran aground, the coast guard said.
"She stemmed away from the area but because where the damage occurred it flooded the propulsion motor room," Jim Chmiel, superintendent of regional operations centre, Atlantic region, told CBC News.
Chmiel said crew closed water tight doors, confining the area were water leaked in the ship.
Twenty-six crew and two cadets were onboard at the time. All are safe onboard the ship, the coast guard said.
The CCGC W.G. George, based in Burgeo, will remove two non-essential personnel and two cadets from the ship. The rest of the crew will stay on for now.
The Canadian Coast Guard is assessing the situation hourly.
The Joint Task Force Atlantic tweeted mid-afternoon that the vessel was taking on water.
A coast guard helicopter and the CCGS Louis S. St.-Laurent, a heavy icebreaker, are heading to the vessel, with the help of the Joint Rescue Centre in Halifax.
The icebreaker should reach the stranded ship around midnight to tow it to safe haven, and will stay there until the CCGS Telost arrives Thursday.
The search and rescue ship CCGC W.G. George, which is based at Burgeo, has reached the Ann Harvey and attached a tow line.
The coast guard said the weather in the area is favourable for the ship's position.