Van Koeverden, of Oakville, Ont., led for most of the race at Eton Dorney before Norway's Eirik Veras Larsen passed him in the final 250m.
Larsen finished in three minutes 26.462 seconds for gold, van Koeverden took second in 3:27.170 and Germany's Max Hoff made the podium with a time of 3:27.759.
"Overall I am happy with my race," van Koeverden said. "I tried to get as much air in my lungs and keep it going. Everybody is out there trying to win the race and I feel like my race plan went well. I was looking for a good finish. I can find the silver lining in silver."
It's van Koeverden's fourth Olympic medal. At the 2004 Athens Olympics, he won gold in the K-1 500m and bronze in the K-1 1,000m, adding silver in the K-1 500m four years later in Beijing.
The 30-year-old was the top qualifier in the semifinals with a time of 3:28.209. Hoff was second (3:29.294) and Larsen was third (3:29.547).
Just minutes after van Koeverden won silver, Burlington, Ont.'s Oldershaw came from behind in his canoe race to capture bronze in 3:48.502.
Germany's Sebastian Brendel won gold in 3:47.176, while Spain's David Cal-Figeroa held off Oldershaw for silver in 3:48.053.
Oldershaw, a third-generation Olympian (his grandfather, father and two uncles also competed in canoe events) and the first in his family to make the podium, gave Canada its 13th medal of the London Games.
"My mom, my dad, my sister, my girlfriend, cousins, uncles, aunts. Everyone is here," Oldershaw said, before breaking off to give his mother, Connie, a big hug.
"I'm so happy to be a Canadian and to be an Oldershaw. To represent both is a great feeling."
Oldershaw and Van Koeverden both train out of Burloak Canoe Club in Oakville.
"Oh man, I'm happy for him," van Koeverden said.Suggest a correction