The incident took place in the early afternoon around the Kitsilano Beach Park.
They say a woman who was walking the pit bull approached the man and the pug, and after a brief introduction the larger dog bit down on the smaller dog's neck.
Police say the man and woman couldn't separate the animals, and the man "put down" the pit bull with a folding knife.
The pug was taken to a veterinary hospital for stitches, and police say they are not considering charges.
Marcie Moriarty of the BC SPCA says officials will perform a necropsy on the dead dog and will interview the individuals involved.