Ford, who as recently as weeks ago promised that he had given up alcohol, has admitted he was drinking and called Monday's restaurant incident a "minor setback."
The YouTube video shows Ford using Jamaican swear words and other profanities, at one point aiming his curses at Toronto police Chief Bill Blair.
Ford offered no apologies on Wednesday and was defiant, saying he is entitled to a personal life, and what he does on his own time doesn't affect his work as mayor.
Ontario Premier Kathleen Wynne said she is concerned Ford's personal problems are drawing attention away from the real issues facing Canada's largest city.
Coun. Denzil Minnan-Wong said he agrees Ford is entitled to a private life, but pointed out that in this case the
mayor was not in a private place. Several councillors said Wednesday that Ford's personal problems have left him unable to serve as mayor.
"This is just a joke that has gone on too long," said Coun. Joe Mihevic. "He needs to understand how much he is hurting the brand, the City of Toronto."