The Field of Possibilities sits near the corner of Bernard Street and De Gaspé Avenue in Montreal's Mile End and has long been used as a neighbourhood park by nearby residents.
According a citizens' group called Les amis du Champ des possibles, the city bought the land from CP in 2006.
The group said the parcels of land are rich in biodiversity, and that residents had been fighting to get the green space protected by the city for years.
Both the city and the Plateau-Mont-Royal borough confirmed to CBC News that CP did not have the right to bulldoze any part of the park.
The workers ripped up plants and grass last week and set up a staging area to build train tracks before moving them.
CP spokewoman Salem Woodrow said the land was used in error, and that workers will remove bulldozers by the middle of next week and pay to refurbish the park.
"We regret any inconvenience," Woodrow said.