TORONTO - Prime Minister Stephen Harper went home Saturday night, back in time to what he called the "small town in Toronto" of his childhood.

Speaking at a gala marking the 100th anniversary of Leaside, Harper reminisced about the east Toronto neighbourhood where he grew up in the 1960s.

He says back then, girls and boys were still segregated in the school yard, although some of the older boys were growing out their hair and the girls started wearing shorter skirts.

Harper says it was a place where the local pharmacist made house calls and you could knock on a neighbour's door if you needed help.

He says he has some political memories too — namely the passionate national debate over a new Canadian flag in 1964.

Harper says emotions ran high and some neighbours even stopped speaking to each other.

So as a five-year-old boy, Harper says he plowed right into the debate and asked everyone on his street which design they favoured and why.

Some people, like his parents, favoured one of the two principal designs for a new flag, he said.

"The Harpers liked the one with the blue borders," Harper said with a smile, "and three maple leaves."

He remembers watching the Maple Leaf going up the flagpole outside his kindergarten class in 1965. "And very quickly, peace came back to the neighbourhood," Harper added.

Leasiders were united on one issue, he said. They fought to keep their independence as a municipality, a battle they lost in 1967 when they were amalgamated with the township of East York.

"In Leaside you were, and I was, part of a community in the most intimate and the very best sense of the term," he said.

The neighbourhood has changed since then and become more affluent, Harper said. His parents sold their home for $39,500 in 1971 — "probably not the wisest financial transaction they ever made," he said. Homes cost more than 10 times that now.

A few years ago, he got off the bus and walked around the neighbourhood where he spent the first 12 years of his life, Harper said. Decades later, much of it was "strangely, almost hauntingly identical," he said.

He could close his eyes and see every street, every yard, every alley, every crack in the pavement the way it was then, except that many of the people are now gone, he said.

"My father, my grandparents who would visit, aunts and uncles, family friends, our older neighbours. As the song says, I loved them all," an emotional Harper said. "But all that love, and so much love, that I still carry with me."

People ask him from all over the country where exactly he comes from, he said.

"And I say to them, I grew up in a small town in Toronto."

Related on HuffPost:

Loading Slideshow...
  • Most Hilarious Harper Moments

    (CP)

  • The Hat

    It seems the hat is balances carefully on the top of the head, rather than being worn. Stephen Harper, far left, watches a third round match between Agnieszka Radwanska, of Poland, and Serbia's Jelena Jankovic with his children Rachel, center, and Benjamin, right, at the 2012 US Open tennis tournament, Saturday, Sept. 1, 2012, in New York.

  • The Vest

    Who can forget this Calgary Stampede ensemble? (CP)

  • The Other Vest

    Harper, centre, chose to wear what amounts to a fishing vest on a trip to Mexico in 2006 (where he visited with U.S. President George Bush (left) and Mexican President Vincente Fox (right)). See how Bush and Fox are decked out in desert duds? Light linen shirts, khakis? Ya, that's what you wear in the desert. (CP)

  • The Headdress

    Harper wears a headdress after being made an honourary chief of the Blood tribe during a ceremony in Stand Off, Alberta in 2011.

  • The Brooding Teen

    Harper was a teenager once too.

  • The Beach Walk

    <a href="http://www.huffingtonpost.ca/2012/04/17/stephen-harper-beach_n_1431449.html" target="_hplink">Harper took an awkward walk</a> along the beach in Colombia on a recent trip. (Getty)

  • If You Were A Vegetable?

    "I would choose to be a fruit, just what I am, sweet and colourful."

  • Harper Does Neil Diamond

  • Harper Does The Who

  • Harper Does The Rolling Stones

  • Harper Does The Proclaimers

  • Harper Does The Guess Who