TORONTO - The city councillor brother of Toronto Mayor Rob Ford did not back down Sunday from comments about a home for developmentally disabled youth in his ward, saying it is a "nightmare" for the neighbourhood.

Coun. Doug Ford told TV station CP24 that his "heart goes out" to families with autistic children, but he said this issue is about kids in the residential home who have "violent tendencies."

"I've been a Rotarian for 25 years helping kids with challenges, but you can't disrupt the neighbourhood like that," Ford said.

"Anyone who wants to criticize, I'd be more than happy to take their address and we'll put the house right next door to them and see how they like it."

Community newspaper the Etobicoke Guardian reported that Ford's office organized a public meeting on Thursday concerning the facility run by the Griffin Centre, described as a non-profit multi-service mental health agency.

The paper reported that some residents in Toronto's west-end complained about the facility's presence in their midst and that they weren't given sufficient advance warning.

Ford was quoted as saying the home should be relocated.

"You’ve ruined the community," Ford reportedly told the facility's staff.

"You can't destroy a community like this. People have worked 30 years for their home...My heart goes out to kids with autism. But no one told me they’d be leaving the house. If it comes down to it, I'll buy the house myself and resell it."

The comments drew condemnation on social media, including former Ontario premier turned Liberal MP Bob Rae, who weighed in with a tweet on Saturday.

"This is the opposite of leadership on mental health. Doug Ford should be ashamed of himself - hurting not helping," Rae tweeted.

Ford shot back at Rae in the interview, saying he "doesn't have a clue what's going on in that neighbourhood."

"Maybe he should go there and get his little, you know, elitist, pompous self over there and maybe we should put a house right beside his house because I know he lives in an absolutely gorgeous area," Ford said.

Ford suggested the root of the problem is the closure under the Liberal government of a regional centre in the west end.

"The provincial government, (Premier) Kathleen Wynne, they closed down the Thistletown Regional Centre," Ford said.

"It was a beautiful centre, had 43 acres that allowed families to have their children with challenges there. Since she has closed that down they have dispersed these folks throughout the west end."

Wynne did not directly respond to Ford's comments when asked about the matter on the provincial election campaign trial Sunday in Mississauga, Ont.

She instead said her government had earmarked $810 million in budget funding over three years to help people with developmental disabilities, and blamed the opposition for shooting down the spending plan. The opposition parties' refusal to pass the budget triggered an election set for June 12.

The services and programs that Thistletown provided were transferred to the community and the closure is in line with government efforts to "build a more co-ordinated mental health system," the province said when it announced the closure.

At that time there were 15 adult and youth residents at Thistletown and about 400 people were receiving support through non-residential programs.

Ford told CP24 the Griffin Centre "misled" him into believing the children there wouldn't leave the house unsupervised. Instead, he said, kids with "violent behaviour" leave the home unaccompanied.

He suggested screaming can be heard in the wee hours of the morning, that cars are being broken into, and that emergency services are frequently called to the home.

"It's turned into a nightmare for this community," Ford said. "It's been turned upside down."

In an email to The Canadian Press, Deanna Dannell, a Griffin Centre spokeswoman, said the centre had several contacts with Ford before the residence opened and it was upfront about who would live there and what services would be provided.

John Tory, who is running against Rob Ford for mayor in the October municipal elections, released a statement Sunday calling Doug Ford's comments "deeply regrettable and from another age."

"For years, it was thought the best way to help people with disabilities, including those with autism, was to place them in large institutions — a kind of confinement away from the community," Tory wrote.

"Today, we know what is best for us and best for them is to include them in every possible way — at school and in our community. For Doug Ford to express surprise that kids with autism would 'be leaving the house' is incredibly out of touch and insensitive."

Doug Ford has announced he won't seek re-election as councillor, but is running his brother's re-election campaign. The mayor, however, suspended his campaign on April 30 before leaving for rehab to deal with substance abuse issues.

Rob Ford was spotted in Ontario's cottage country ahead of the long weekend. The CBC reported that Ford's doctor said the mayor was still in rehab and his off-property jaunt was approved and supervised.

Loading Slideshow...
  • Councillor Doug Ford and his brother Mayor Rob Ford gets into a shouting match with members of the public in chambers at City Hall in Toronto on November 18, 2013 during a special council meeting to limit more of his powers. (Deborah Baic/The Globe and Mail)

  • Toronto City Councillor Doug Ford makes his way to the council chamber as councillors look to pass motions to limit the powers of his brother, Mayor Rob Ford, in Toronto on Monday November 18, 2013.THE CANADIAN PRESS/Chris Young

  • Councillor Doug Ford reads a prepared statement to the media outside his brother Rob's office at Toronto city hall on May 22 2013. Rob Ford, the mayor of Toronto, is at the centre of a scandal stemming from a video allegedly showing him smoking a crack pipe. (Fred Lum/The Globe and Mail)

  • Toronto city councillor Doug Ford leaves a radio station where he has a program with his brother, Mayor Rob Ford, in Toronto on Sunday May 26, 2013. THE CANADIAN PRESS/Aaron Vincent Elkaim

  • Toronto City Councillor Doug Ford reads a statement to journalists at Toronto City Hall on May 22, 2013. The man who has most strongly defended Toronto Mayor Rob Ford against allegations he was filmed smoking crack cocaine is vehemently denying a claim that he himself once dealt in hashish. The Globe and Mail alleges in a story published Saturday that the mayor's brother, Coun. Doug Ford, was once a drug dealer in '80s -- a claim denounced by his lawyer as false. THE CANADIAN PRESS/Chris Young

  • City councillor Doug Ford defends his brother, Mayor Rob Ford, at city council in Toronto on Wednesday, Nov. 13, 2013. THE CANADIAN PRESS/Nathan Denette

  • UP NEXT: Renata Ford, Rob Ford's Wife

  • Renata Ford attends a news conference as her husband Toronto Mayor Rob Ford speaks to the media on Thursday November 14, 2013. THE CANADIAN PRESS/Chris Young

  • Mayor Rob Ford's wife Renata Ford, centre, is escorted to her car by staff after attending a press conference with her husband at city hall in Toronto on Thursday, Nov. 14, 2013. THE CANADIAN PRESS/Nathan Denette

  • Mayor Rob Ford's wife Renata Ford, second left, is escorted to her car by staff after attending a press conference with her husband at city hall in Toronto on Thursday, Nov. 14, 2013. THE CANADIAN PRESS/Nathan Denette

  • Toronto Mayor Rob Ford stands at an elevator door as he tries to escort his wife Renata out of a news conference on Thursday November 14, 2013. THE CANADIAN PRESS/Chris Young

  • Toronto Mayor Rob Ford (centre) jostles with the media as he tries to escort his wife Renata (left) out of a news conference in Toronto on Thursday November 14, 2013. THE CANADIAN PRESS/Chris Young

  • Rob Ford on election day at his family's (mom and dad's) home in Toronto on October 25, 2010, watching as the results come up on the TV announcing him as the winner, and shortly afterwards, hearing that his brother Doug won as well. Hugs from his mother, Diane, and his wife Renata.This is when he found out his brother had won as well. Renata, Rob's wife, threw her arms up, as did mom, Diane, at left. (Photo by Peter Power/The Globe and Mail)

  • Mayor-elect Rob Ford being greeted by his supporters and making his speech at theToronto Congress Centre in Toronto on October 25, 2010, after winning the election. This is where he said the win was for his father. Mom, Diane, is left and wife Renata is right. (Photo by Peter Power/The Globe and Mail)

  • Rob Ford on election day at his family's (mom and dad's) home in Toronto on October 25, 2010, watching as the results come up on the TV announcing him as the winner, and shortly afterwards, hearing that his brother Doug won as well. Hugs and kisses immediately followed with his wife Renata. (Photo by Peter Power/The Globe and Mail)

  • Mayor Rob Ford gets a kiss from his wife Renata at city hall council chambers as he was officially sworn in as new mayor of Toronto Dec. 7, 2010.(Photo by Kevin Van Paassen/The Globe and Mail)

  • UP NEXT: Twitter Reaction To Ford's Comments

  • Next: Rob Ford's Biggest Gaffes

  • "Assgate"

    <a href="">In March, former mayoral candidate Sarah Thomson accused Rob Ford of grabbing her behind</a> while at an event. She <a href="">later suggested the mayor was so out of it that she wondered if he was on cocaine. Ford has denied the accusations.

  • Goodbye, Hello

    <a href="">In November of 2012, an Ontario Superior Court Judge ordered Rob Ford to be removed from office</a> for violating Toronto's Conflict-of-Interest Act. The ruling stemmed from Ford's participation in a council vote to recommend he repay donations that he solicited for his private football foundation using official city letterhead. After weeks of uncertainty about who would replace Ford, the <a href="">mayor won his appeal</a>, allowing him to remain in power.

  • Football Follies

    <A href="">Ford's role as coach of a high school football team</a> has repeatedly landed him in hot water. From a <a href="">city bus used to ferry the team home</a> after a reported brawl, to <a href="">missed council meetings and court appearances</a>, Ford's gridiron exploits have made headlines again and again. Despite the controversy, Ford has maintained that he's not giving up his other job to focus on running the city.

  • Winnipeg, Windsor, What's The Difference?

    On a trade mission to Chicago, <a href="">Ford infamously confused Winnipeg and Windsor</a>, a verbal stumble that <a href="">prompted chuckles on both sides of the border</a>.

  • Falling Down

    <a href="">A video clip of Ford falling</a> while attempting to throw a football at a Grey Cup event was quickly turned into GIF image that went viral.

  • Driving Mr. Ford

    In October of 2012, a photo hit the web of <a href="">Ford reading while driving</a>. The mayor admitted to doing it, but <a href="">refused to hire a driver</a>, despite pleas from the police and political allies. The incident was far from the first Fordian fail on the road. The mayor has also been <a href="">accused of giving a motorist the finger while driving</a> and has admitted that he <a href="">pleaded guilty to refusing to provide a breath sample</a> after driving under the influence of alcohol in Florida.

  • Unfortunate Photo

    A photo hit the web in September of 2012 of <a href="">Ford posing with a neo-Nazi</a> dressed in a foreign military uniform. Ford explained that he was unaware of the man's political beliefs. At least <a href="">one major Jewish group said it was satisfied the mayor meant no harm</a>.