Speaking at the YWCA in Vancouver, Rae said the relationship of governments with aboriginal people "is a great example of truly unfinished, unreconciled business in the country."
The wave of aboriginal "immigration" from reserves during the last 50 years means the poverty they face is no longer hidden, he said Saturday.
Rae said many of Canada's largest cities now include "aboriginal ghettos" where the anger fuelled by inequality has led to suicide and crime.
"Toronto has been described, quite rightly, as the largest reserve in the country," he said. "This problem is no longer out there, this problem is right here.
"As it gets closer to us it also has to be closer to our hearts and to our understanding, to extent that we have not embraced aboriginal culture, we have not understood First Nations history, we have not appreciated the loss, the indignity and the deep sense of injustice that now has to spur us to action."
The former NDP premier in Ontario said the federal Conservative government's response has been to build more jails but that hasn't increased safety in communities or reduced crime.
Younger generations of First Nations aren't getting the education they need to get ahead and many are living in foster homes, he said.
"There are more kids today across the country who are in foster care and being taken away from their homes than during the residential community time," he said.
But he said South Africa, Australia, New Zealand and the United States also deal with the same issues regarding their aboriginal people and no country has come up with the right policy.
Better social policies are also needed to address housing, which was abandoned as a priority by Ottawa and most provincial government 25 years ago, Rae said.
"Each level of government says,'It's not my responsibility, somebody else has to look at this,'" he said, adding that some cities have taken a leadership role to deal with the issue.
Rae said that when the Conservatives came to power in 2006, their first budget identified housing as a provincial issue with too much federal government involvement.
But he said Ottawa must be a reliable partner in housing because it's indirectly involved in interest rates and mortgages and can't pretend it doesn't have any expertise or resources to deal with it.
However, he said the federal government can't go back to the expensive programs of the past and must instead involve the private sector.
Rae also said Canadians desperately need better social policies for mental illness.
Every province has shut down its largest mental health institutions, but programs to replace them aren't adequate, Rae said.
Even soldiers and civil servants who served their country in Afghanistan and are suffering from post-traumatic stress aren't getting the help they need, he said.
"We, as a party, need to be the ones to say, `How are we going to deal with this problem?'"