Instead, they mostly opted to attack him on local issues, including his plans for a subway, unemployment and the frequent disarray he has caused at city hall.
Ford, who at times rolled his eyes, stared at the ceiling and flashed grins at the audience, defended his fiscal record as he shot back at the other four candidates.
"Four years ago, I was elected to stop the gravy train," Ford said, thumping a familiar drum.
"I have stopped the gravy train."
Former NDP member of Parliament, Olivia Chow, was quick to shoot him down.
"It's time for you to go because your gravy train has turned into a train wreck," Chow said.
"You need to pack up that nonsense you're talking about."
Chow was the first candidate to allude to but not mention Ford's well known problems — including smoking crack cocaine in a drunken stupor and spouting obscenities on live television — that have elevated him to TV talk show fodder and made him a recognizable name around the world.
"We've had it with the scandals and lack of truth," she said. "It's time to go because you're embarrassing our city."
Ford did not bite. Instead he insisted that he was the only candidate with a "proven" track record.
"People have heard the story. It's rewind, rewind, rewind," Ford said.
"You can carry on about someone's personal life. Maybe I'm not perfect."
Asked about their hesitation to tackle Ford directly on the scandals, the candidates said they preferred to talk policy.
"My big disappointment with Rob Ford, even beyond the scandals, is with his performance," said David Soknacki, a former city budget chief, after the debate.
"I'm dealing with the public side of it and am very much dealing with the issues of holding him to account for his lack of performance."
American TV comedian, Jimmy Kimmel, who has frequently made fun of the mayor, tweeted at one point that Ford was "JFK compared to some of these candidates."
Ford, his face red, was often content to let the others debate among themselves, but didn't hesitate to jump in.
"Everybody can talk about saving money: I've done it," he said. "We don't need a left-wing tax-and-spend NDP government."
When Chow, a former city councillor, talked about her work on the budget committee, Ford shot back: "You sunk the ship."
Former provincial Progressive Conservative party leader, John Tory, said Ford had "run out of gas" given his inability to get anything through a council that has stripped him of most of his powers because of the scandals.
The businessman accused the mayor of letting taxpayers down and ruining the city's reputation. Ford simply said he didn't expect that of his opponent.
Mostly, though, it was Chow and Ford who went at each other during the two-hour debate.
"It's time to take down the circus tent at city hall," Chow said, saying the mayor had made Toronto "an international embarrassment."
Voters will choose whether Ford remains in office or will be replaced Oct. 27.
Also on HuffPost