But the blind, 74-year-old competitor at the International Federation of Disabled Sailing world championships this week in Halifax says he didn't hesitate when offered a chance earlier this year to bid for a spot at the Rio Paralympic Games.
"The same good world is still out there and you have to experience it in a different way," said Kerr in an interview at the Royal Nova Scotia Yacht Squadron site.
The former alternate member of the 1964 U.S. Olympic modern pentathlon team and Olympic fencer for the U.S. Virgin Islands, said when he lost his sight in his 60s, he thought he would remain a recreational athlete.
"I certainly never expected to again reach the level I had experienced in some other sports," he said.
"But when I was approached by the folks in Virgin Islands to see if I'd be interested in joining the sailing team it took me about a millisecond to say, 'Sure.'"
Kerr says he's discovered there is an emerging role for blind sailors on the Paralympics, as they operate lines and rigging aboard the three-person Sonar vessels racing here.
"I'm happy to say to other blind people that this is a glorious thing to experience. You can experience it just as much as if you had sight," he said.
Kerr said he has been inspired by Nicolas Vimont-Vicary of France, a member of the reigning world champion team in the Sonar class.
The two met on Sunday during preparations for the event, and within minutes Vimont-Vicary offered the American the chance to try a sensor technology helmet that tells him the wind direction.
The technology allows the blind sailor to take control of his own vessel, and participate in blind-only sailing competitions as a skipper.
"I would like to show him (Kerr) so that so he can take the helm too," he said during an interview.
During the Sonar championships, both Kerr and Vimont-Vicary say their key challenge is to listen for each command, react quickly during turns, and to shift their weight quickly to improve boat speed.
"I have to concentrate," says the French sailor. "Even if I don't see what's happening, I have to hear everything going on around the boat."
Vimont-Vicary counts as he wraps the jib sheet around his hand during a turn, while Kerr tends the traveller that shifts the boom across the boat.
Each team at the championships goes through detailed assessments by medical teams of their various disabilities and must meet a common international standard before they are permitted to race one another.
The U.S. Virgin Island's team's Dave Flaherty has suffered a stroke and has some paralysis on one side, while Kerr's fellow crewmate Bob Blackwell does his work on the sheets with one arm.
Each is graded on the extent of the disability by the competition's medical experts.
John Foster, the 76-year-old coach of the Virgin Island team, has competed in five summer games in sailing and once in winter Olympics in the bobsleigh.
"This is giving back a little bit of all the joys and experiences you've had in your life, trying to help others enjoy it as much as you did," he said as his vessel was launched.
The competition begins for the Scud and Sonar classes on Tuesday.
As Kerr prepared to compete, he said his hopes were high for a top-seven finish.
"I'm mostly learning, I'm mostly listening. ... But we all have a great spirit and that's another part I'm happy to be part of."