Premier Darrell Dexter signed the agreement in Halifax with Peter Kent, the minister responsible for Parks Canada.
The crescent-shaped island sits about 300 kilometres southeast of Nova Scotia and is home to 400 wild horses, as well as breeding grey seals and numerous bird species.
The island, which is 42 kilometres long, is also known as the Graveyard of the Atlantic for the more than 200 wrecked ships off its coastline.
Nova Scotia and Ottawa will table legislation to prohibit oil and gas drilling from the surface of Sable Island and out to one nautical mile from shore.
Dexter said Sable Island holds a special place in the hearts of Nova Scotians and he's pleased it will be protected as a park reserve.
"The mystique of Sable has always been a part of Nova Scotia's identity," he told a news conference Monday at the Halifax Citadel National Historic Site.
"Images of those Sable horses with their manes whipped back in the Atlantic winds are ingrained in our collective memories. In fact, they're on the walls of my house."
Kent said the agreement ensures protection of the island's unique character and landscape.
"At the same time, we will offer Canadians opportunities to discover, appreciate and understand this iconic treasure," he said.
Plans to protect Sable Island have long been in the works.
Ottawa received recommendations in 2009 to add the island to its list of national parks. In January 2010, the provincial and federal governments committed to pursuing status for the island as a protected area.