The security breach happened on Sunday evening and was captured on a cellphone video by a 25-year-old man who was visiting the zoo with his fiancee.
Andrew Fedurko said he accidentally walked into an off-limit area, assuming it was part of the exhibit.
"We walked into an enclosed area to the right of the cage," he said, adding that his fiancee was leaning against the chain-link fence when a panda suddenly appeared from around a corner.
"It didn't look like it was going to jump at us necessarily but I'm not a panda expert," he said.
Fedurko said once he realized how close they were to the animal, they stepped back and he captured other visitors on video.
The footage shows visitors leaning up against the chain-link fence, with some putting their hands up against the fence while the panda is nearby.
Da Mao is one of two prized giant pandas on a 10-year breeding loan from China as part of a conservation partnership. The pandas will be in Toronto for a five-year term, set to end in 2018, at which point they will be relocated to the Calgary Zoo.
A zoo spokeswoman said visitors had bypassed a bamboo barrier — approximately 4.5 metres high — that been closed and latched but there was no padlock on it, which is zoo protocol.
"It appears that a visitor would have undone the latch and opened the door," Jennifer Tracey said in an interview, adding that a preliminary investigation suggested the gate was open for between 15 and 25 minutes.
Fedurko said prior to recording the video, children near the chain-link fence had taken pieces of long grass and tried to poke the panda.
"We were very uncomfortable recording a video where we were expecting a child maybe to be attacked, it didn't sit well with us," he said, adding that he went to find zoo staff. "There was no staff anywhere in sight."
Tracey said there are two staff members dedicated solely to the giant panda exhibit on hand to answer questions from the public.
"From my understanding there was the dedicated staff around at that time," she said.
But Fedurko said he had to walk to the panda gift shop to report the problem, and then walk further to guest services.
Tracey confirmed that a call reporting the open gate came from the panda gift shop, and said part of the investigation will involve looking at where staff was stationed during the incident.
She said once the zoo was made aware of the problem, it took three minutes to respond.
Tracey added that security protocols for the pandas include an electrified wire running across the top of the chain-link fence, in case they try to climb out of the enclosure.
There are also multiple security cameras and the zoo will be reviewing footage in their investigation, she said.
"There's a reason why we have a double barrier in front of the exhibit, it's not only to protect our visitors but to protect our animals themselves," she said, adding that pandas are critically endangered and can be dangerous.
"They are in fact bears. They have teeth seven times the size of human teeth and they have claws," she said. "Even our giant panda keeper staff all have protected contact from the animals themselves."
Tracey said she has viewed a video of the incident, and was "disappointed."
"You can see there were adults putting small children in front of the fence. It's clearly an area that is not maintained, it's not part of the exhibit area, you have to climb over a boulder to get into it," she said.
Tracey said there are no signs outside the bamboo barrier because "you wouldn't even notice it's a gate."
She added that after the investigation is complete, the zoo will review any recommendations to ensure visitor safety.
"We believe this to be quite an isolated incident," Tracey said. "It's very unfortunate that it happened and we are grateful that nobody was hurt."
In May, the zoo investigated after two high school students jumped a fence to take photos with a five-month-old polar bear cub, getting close enough for the cub to lick their hands.