Recent reports by the Toronto Star and the U.S. gossip website Gawker allege that a video exists of the mayor smoking from what appears to be a crack pipe.
Ford has said the allegations are both untrue and "ridiculous," though some councillors have called on the mayor to provide more substantive comment on them.
Coun. Josh Matlow said Sunday that the public needs a more substantive comment from the mayor on the allegations.
"I think that the story needs to come out," Matlow said during an interview with CBC News.
"It's a shame that the mayor hasn't confronted the allegations directly beyond a one-word answer and I think that for his own sake and for the city’s sake, we need to put this behind us."
Matlow and Coun. Kristyn Wong-Tam both told CBC News this weekend that the mayor needs to leave office if the allegations are true.
"The allegations that have been leveled at Mayor Ford are not just serious, but they are also criminal," Wong-Tam told CBC News in an interview.
"I would encourage the mayor to resign and to seek help as soon as possible should this be true."
Matlow said that "if the allegations are proven true beyond a reasonable doubt, then the mayor should resign" from his job.
On Friday, Ford did not answer questions from reporters at city hall, though he did offer a brief comment on the report in the Star.
"Anyways, like I said this morning, these allegations are ridiculous, it’s another story with respect to the Toronto Star going after me," Ford said. "And that's all I’ve got to say for now."
CBC News has not seen the video and has not been able to validate any of the claims being made.
Ford won't be on air today
No further comment is expected from Ford on Sunday, given that he is not going to be appearing on the two-hour radio show he normally hosts with his brother, Coun. Doug Ford.
Coun. Ford told Newstalk 1010 yesterday that he had "never" seen the mayor "involved with anything like coke."
The controversy flared up just ahead of the holiday weekend, when Gawker and the Star also revealed that someone had been trying to sell the recording to Canadian and U.S. media.
While Gawker did not purchase the video initially, it has launched a campaign to collect donated funds that it plans to use to purchase the recording from the unidentified person or people who have been shopping it.
As of Sunday morning, Gawker said it had raised more than $60,000. The website has said it needs $200,000 to obtain it.
Ford, who turns 44 later this month, has served as Toronto's mayor for the past two-and-a-half years. He previously served as a councillor in Etobicoke, the Toronto suburb where he lives with his family.
Through his time as mayor, Ford has struggled to lead a council that has been divided on many key issues, including taxes and transit.