They say developers are buying up older properties in Woodward's Hill neighbourhood, and then letting them sit unoccupied while they wait for them to be demolished.
Michael Gibbs, one of the frustrated residents, said the abandoned homes are an eyesore and a magnet of crime for the rest of the tree-lined community.
"The abandoned homes have become areas of crime, and we suspect, because we've actually followed some of the petty theft issues into these houses," he said.
"If this was West Vancouver. If this was White Rock, if this was Morgan Creek, right in Surrey, they wouldn't tolerate this for a second. Those houses would be removed."
Developers, such as Sanver Shoker, say squatters are a major concern for them, too, and keeping them out isn't easy.
"You can board up a property, they'll go to the back and take stuff down and you might not know for a couple of weeks," Shoker said.
Surrey bylaw manager Jas Rehal said all the city can do is ensure the property isn't a safety hazard.
"We can make contact with the property owner and ask them to board it up. If they don't, then we'll move in and board the house up," he said.
But that's not enough, said Gibbs, who added the city must crackdown on the boarded-up homes and force the developers who own the abandoned properties to do something about it.