For one, his country didn't have much of an international pedigree in the sport. And he was a black man from KwaZulu-Natal province, where sporting prowess was traditionally confined to rugby, cricket, football and swimming.
So when Ndlovu realized he had stroked South Africa's lightweight men's four to a 0.25-second victory Thursday over the favoured British to win South Africa's first gold medal in the sport, he leapt into the arms of each of his crew, not quite believing the names next to the No. 1 position on the big TV screen.
The win will make Ndlovu and fellow crew members James Thompson, Matthew Brittain and John Smith iconic figures back home.
"I am the first black man in South African rowing (to win gold)," he told The Associated Press. "I feel very proud of that and for people in Africa to see what I've been doing."
It will also mean a royal reception from the Zulu tribal king in his home province when he returns.
"He will be received as a prince or a king," South African chef de mission Patience Shikwambana said. "We call KwaZulu-Natal 'The Kingdom.' So that means when he gets there, the king is going to come and welcome him and say, 'Yes boy, you've done us proud.'"
Shikwambana said he hoped the 31-year-old Ndlovu, who is due to start a degree in sports science after the Olympics, will inspire a new generation to take up rowing instead of traditional sports like soccer.
"People normally ask you, 'Why do you do rowing?'" Ndlovu said. "It's costly and it takes up a lot of time. You can't just train for a month and be done. It's a full-time commitment. There's no money, and you have to work so hard at the same time."
Before 1990, there would have been no chance of Ndlovu being picked for South Africa's Olympic squad.
Apartheid existed in sports as much as any other walk of life, meaning that blacks were racially segregated and only whites were generally selected for international competition. That had all changed by 1994 when apartheid officially ended after the African National Congress under Nelson Mandela won multiracial democratic elections.
In 1995, Chester Williams played for South Africa's rugby union team that won the World Cup on home soil. Williams was a beacon for black youth in sports, and Ndlovu's gold medal may have the same effect.
He started rowing 15 years ago and has been supported by the South African federation.
"In the past two years, I've made really important strides with the three guys standing behind me," he said, referring to his crew. "They mean a lot to me."
Ndlovu isn't the only African to make a big impact at the rowing regatta at the London Games.
Despite finishing last in his three races, Niger sculler Hamadou Djibo Issaka captured the hearts of spectators with his determination despite having only three months of experience in rowing.
He was given a wild card for the games from the IOC Tripartite Commission, which allows each National Olympic Committee to enter up to five athletes for the Summer Games. FISA, rowing's governing body, is hoping to raise the profile of the sport in landlocked Niger by providing equipment such as professional boats and oars.
The win by South Africa meant disappointment for Britain, which had qualified quickest from the semifinals.
Britain's Peter Chambers told BBC Radio after the race that he was angry his boat hadn't been allocated an outside lane, which may have meant they would have been affected less by the crosswinds.
"The second kilometre got a lot more fair and that's where we got our speed up," Chambers said. "FISA should be ashamed that they didn't change that in the finals."
Chambers later said his comments were made in the heat of the moment and that Britain had no plans to make a complaint.
Denmark, led by triple Olympic gold medallist Eskild Ebbesen, almost delivered a wire-to-wire victory after taking the initiative early on. That would have been the country's fourth victory in five Olympic finals for the lightweight four.
But the Danes faded in the final 100 metres, allowing South Africa to charge through. Denmark won the bronze, just 0.07 seconds behind Britain. It was the fifth career Olympic medal for the 40-year-old Ebbesen.