Police were called around 12:45 p.m. and arrived to find a 37-year-old male who is well-known to police and who suffers from mental instability.
Cpl. Bert Paquet said in a statement that the man was holding the child and had a knife in his possession.
Officers talked the man into letting the child go, and he was arrested after being shocked with a conducted energy weapon.
Police say the child was not injured.
Officers are looking into whether there was any connection between the man and the child.