The piano in Notre-Dame-De-Grace Park on Girouard street had only been in the park for a week as part of the city of Montreal's Piano in the Park initiative.
An inauguration event for the instrument was planned for later this week.
The event is now cancelled and the piano has been removed to see whether it can be salvaged.
NDG resident, Mac Isaac, said he had come to the park to play the piano a few days ago.
"A lot of kids came around when I was playing it. It's just terrible," said Isaac.
The NDG borough has one other public piano in Côte-des-Neiges, which officials say they have no plans to remove until the end of the season later this month.
Street pianos first appeared in the Montreal's Plateau two years ago, however this is the first time any of them has been targeted by vandals.