The Mounties said they wrapped up the search at 6 p.m., nearly six hours after the man went into the water.
Police said an RCMP underwater recovery team will be deployed once sea conditions permit.
The man's name has not been released pending the notification of family.
Police said the man, who is from Ontario, and a woman he was travelling with were walking along the water's edge when he got too close and a large wave came over and swept him into the ocean around 12:15 p.m.
The Mounties said the extensive search included help from a Joint Rescue Co-ordination Centre helicopter and three coast guard vessels.
Cpl. Greg Church said at the time of the incident, there was light fog and the ocean was choppy.
He said there have been similar incidents at the popular tourist destination over the years, but he hasn't encountered one since he joined the Halifax district Mounties seven years ago.
"The risk is always there. It's a risky area to sightsee in," Church said of Peggy's Cove, home to an iconic white and red lighthouse.
There are signs posted at Peggy's Cove warning visitors of the risks of walking too close to the water's edge.