The handshake came during a ceremony focused on Mandela's legacy of reconciliation. The U.S. and Castro have been enemies since the Cuban revolution led by Castro's younger brother Fidel in 1959.
Obama was greeting a line of world leaders and heads of state attending the memorial in Johannesburg just before he spoke to the gathered thousands at FNB Stadium in Soweto.
The U.S. and Cuba have recently taken small steps toward making peace in recent years, raising hopes they could be on the verge of a breakthrough in relations.
Still, skeptics caution that the two countries have shown signs of a thaw in the past, only to fall back into old recriminations.
Canada, for the most part, has maintained warm relations with Cuba since the days of former prime minister Pierre Trudeau's visits to the country in the 1970s. He and Fidel Castro, Raul Castro's older brother, remained on good terms.
Obama also shook hands with Brazilian President Dilma Rousseff, who followed him on the podium and has clashed with Obama over alleged National Security Agency spying.