Kelly, who is Premier Kathleen Wynne's point-man for city affairs, said he left at noon on Christmas Day to visit his last living sibling and returned on the "first flight out" earlier today.
"Quite often in politics you ignore your family," said Kelly. "Over the past several years I've lost two brothers and a sister. My remaining sister is ailing and I just wanted to make contact with family."
As the 72-year-old Kelly travelled to sunny Tampa, thousands of his constituents in his ward of Scarborough-Agincourt went another day without heat and hydro.
As of late Thursday night, Toronto Hydro confirmed there were 46,000 customers without power, down from 300,000 after the storm hit overnight last Saturday.
When asked by CBC Toronto if he regrets the decision, Kelly said he regrets the controversy his trip caused but he "would have gone to North Bay. It didn't matter where it was.
"I understand the optics of it, at the same time you try to live a balanced life and I tried to balance the family side of my life with my obvious obligations to the residents of Toronto," he said.
The decision to travel south was met with mixed emotions from those calling into local Toronto radio shows, or voicing their opinions online.
Answered emails, responded to texts
Kelly said he kept in touch with city officials and residents while he was away.
"I was in touch with the emergency committee. I answered all the emails that came into me during that time. Responded to all the texts and that's in essence how I continued to do my business," he said.
Despite being at all the city briefings about the storm since Sunday, Kelly was absent at Wednesday's 11 a.m. news conference - the same day he travelled to Florida.
"I'm not into press conferences as some other people are," he said. "I prefer my contribution to be solid and important."
Toronto Mayor Rob Ford was grilled at a Toronto Community Housing barbecue at 400 McCowan Avenue, near Eglinton Avenue E., on Thursday where residents remained without power.
With cameras rolling, one angry resident told the mayor: "You better do something fast ... I'm upstairs freezing my ass off, Mr. Ford."
Ford has maintained there is no reason to issue a state of emergency as hydro crews from as far away as Sault Ste. Marie, Windsor, Ottawa and Manitoba work around the clock.
Part of the powers handed to Kelly by city council would be to lead the city if a state of emergency were to be called.