Ford gave a short speech honouring the military and there was a quiet smattering of boos and cries of "shame" in the crowd as he walked up to the podium, though there was also light applause.
After he laid a wreath at the foot of the cenotaph at Toronto's Old City, the mayor walked past a row of veterans and at least one refused a hand shake.
Tony Smith, who was stationed in Germany after the Second World War, said after the ceremony that Ford should not have been there, given his admission last week that he had smoked crack cocaine.
"I didn't shake his hand because he's a drug addict, a druggie," Smith said. "He's No. 1 man in the city and he's smoking up and boozing it up. I don't mind people having a drink. I certainly don't agree with drugs."
Ford reversed five months of denials last week when he admitted he had smoked crack cocaine, likely in one of his "drunken stupors." The Toronto Star and U.S. website Gawker reported in May on the existence of a video appearing to show him smoking the drug and Toronto's police chief recently announced that investigators have seized it.
Ford has repeatedly denied he is an addict.
Meanwhile, his crack smoking admission had Don Cherry, one of his most outspoken supporters, saying Monday he is "disappointed."
When Ford came to power Cherry proclaimed he would be "the greatest mayor this city has ever seen."
"Put that in your pipe, you left-wing kooks," Cherry said at Ford's official swearing-in ceremony at city hall in 2010.
Calling Ford's opponents "left-wing pinkos," Cherry said the mayor was "honest" and "truthful.''
"What you see is what you get. He's no phoney."
Cherry's take on the mayor had notably changed Monday.
"Yes, I am disappointed," he said in a radio interview on Sportsnet 590 The Fan. "If you could just cut out all the rest of the stuff as the mayor he's done a great job and everybody thinks that."
At the Remembrance Day ceremony, hundreds of people braved the rain and Ford shared an umbrella with federal Transport Minister Lisa Raitt. Seated next to her was NDP Leader Tom Mulcair, who has called on Ford to resign and get help.
After Ford laid a City of Toronto wreath, Smith said a fellow veteran seated next to him also refused to shake the mayor's hand. However, at least two other vets could be seen shaking Ford's hand.
Some veterans did not want to speak about the Ford controversy.
Leonard Pelletier, 90, spent the Second World War in England and later served as a peacekeeper in Egypt, France and Belgium.
In the military, he said, "you never discuss politics or religion."
"I don't get involved with politics," Pelletier said. "I don't think about it. He has his thing to do and we all have our own thing to do."
As soon as the Remembrance Day ceremony ended, Ford walked back to city hall, flanked by staffers, his new driver and a city hall security guard, and did not answer reporters' questions.
The mayor's previous week involved shocking confessions of drug use, drunken stupors, a new video showing erratic behaviour and pleas from both friends and foes to step aside to get help.
This week could see even more twists and turns.
The so-called crack video will be in a Toronto courthouse Tuesday as a judge considers an application from lawyers for a man seen in a notorious photo with the mayor.
Ford is seen posing in the photo with Anthony Smith, who was later shot and killed, as well as Mohammad Khattak and another man, who have both been charged as alleged gang members in Project Traveller, a sweeping drugs and weapons investigation.
Lawyers for Khattak were in court Friday seeking access to the video, arguing their client's reputation is being harmed by being associated with it.
They want to either be able to view the video or have Ontario Superior Court Judge Ian Nordheimer view it himself and write some sort of judicial summary of it. Nordheimer ordered the Crown to bring the video to court on Tuesday, when he is set to make his decision.
Nordheimer is also expected to rule this week, as early as Tuesday, whether remaining parts of a document that shed light on Ford's relationship with Alexander Lisi, an alleged drug dealer who is also his friend, can be released.
Lawyers for the media are also expected to argue in the coming weeks that wiretap information from the Project Traveller investigation should be made public.
Toronto Police Chief Bill Blair has said the video of the mayor appearing to smoke crack cocaine was found on a laptop that was seized during Project Traveller.
On Wednesday, a city councillor's motion is expected to be tabled calling on Ford to take a leave of absence, though it will be non-binding. Coun. Denzil Minnan-Wong has said he will ask council to petition the province to oust Ford if he doesn't take a leave.