Abrahams, who was one of many attending a Friday evening memorial to the anti-apartheid icon at the city's art gallery, said he was just 11-years-old when the historic encounter happened.
Abrahams said he and his teammates were playing soccer in a Johannesburg neighbourhood, but Mandela and members of the ANC wanted the pitch for a political rally.
"So they came and told us, 'hey kids, get away.' And I said, 'no, you can't take my sports field away. I've got a game to play,'" said Abrahams who now heads the Vancouver campus of Fairleigh Dickinson University.
"At this time, at 11, obviously I didn't appreciate what the African National Congress stood for."
Abrahams now does, though, so much so, in fact, that he, like many others, surrendered his Friday evening to pay tribute to Mandela.
Under the cold winter sky, some well-wishers huddled near propane heaters outside the art gallery, others wore South African flags as capes, but most just dressed in warm winter clothing, listened to speeches and music and chatted about past memories.
Johannes Nonyane, president of the South African Cultural Association of British Columbia, said he and fellow expats wore yellow armbands to identify themselves to well-wishers.
Nonyane, 45, said he was a child in South Africa when Mandela was imprisoned.
Nonyane who now lives in Surrey, B.C., said he and his friends would visit Mandela's vacant home for inspiration.
"The house was empty. Nobody. It became like a museum for us," he said. "When we wanted to talk politics, when we wanted to talk quietly about what we know, about the struggle, that was an inspiration.
"We went there to look at that house because it symbolized what was wrong with the system in South Africa."
Nonyane said he and his friends also listened to lectures about Mandela from grandparents, mothers and sisters, since that history wasn't taught in the schools.
Meantime, Abrahams said left South Africa in the 1960s and travelled through and lived in a few African countries before settling in Canada as a student and activist.
While in exile, Abrahams said he joined the ANC, and he said he also educated and rallied Canadians against the apartheid regime.
He also pursued his education.
According to his biography posted on the Fairleigh Dickinson University website, he earned his master’s degree in English at the University of New Brunswick in 1965 and PhD in English romantic literature from the University of Alberta in 1977.
He then taught English at the University of New Brunswick and Bishop’s University in Quebec, was academic vice-president and provost at Acadia University and served at Brock University.
His biography states he returned to South Africa in 1995 and became vice-chancellor of the University of the Western Cape.
Abrahams said Mandela was the one who inaugurated him into his university position, and during their meeting he told the South African president about the incident on the soccer field.
"He had a big laugh, and he put his hand out and said, 'let me shake your hand.' which I did (and said), 'I'm not going to wash it for a whole week," said Abrahams.
Mandela died Thursday at the age of 95, and a state funeral is planned for next weekend.