Officers responded to the 3100-block of Douglas Street at 4 a.m. PT after reports of screaming.
Once they entered the room, police found a distraught woman, a young child and a toddler.
Police say the toddler had no vital signs and despite life-saving attempts, the child could not be revived and was pronounced dead at the scene.
The woman and the other child also required medical attention.
In a tweet, Victoria Police wrote "a sad day" in providing details of the incident.
The force, the Vancouver Island Integrated Major Crime Unit [VIMCU] and BC's Coroners Service are all investigating the death.