He achieved it Thursday when he stroked South Africa's lightweight men's four to victory in a nerve-jangling final at Dorney Lake, coming through late to edge a favoured British crew by 0.25 seconds. Denmark took the bronze.
It secured his country's first-ever gold medal in the sport and will make him and fellow crew members James Thompson, Matthew Brittain and John Smith iconic figures back home.
South Africa was a country with precious little international rowing pedigree when Ndlovu gave up his day job. At that point, it had won one medal in rowing — a bronze in the men's pair at the Athens Games in 2004.
Then, there was the fact he was a black man from KwaZulu Natal, a province whose sporting prowess is traditionally confined to rugby, cricket and football.
"I am the first black man in South African rowing (to win gold)," he told The Associated Press. "I feel very proud of that and for people in Africa to see what I've been doing.
"People normally ask you, 'Why you do rowing?' It's costly and it takes up a lot of time. You can't just train for a month and be done. It's a full-time commitment. There's no money, and you have to work so hard at the same time."
When Ndlovu realized South Africa had won Thursday, he leapt into the arms of each of his crew, not quite believing the names next to the No. 1 position on the big screen.
A royal reception awaits for Ndlovu when he return home.
"He will be received as a prince or a king," South Africa team chief Patience Shikwambana said. "We call KwaZulu Natal "The Kingdom." So that means when he gets there, the King is going to come and welcome him and say, 'Yes boy, you've done us proud.'"
Shikwambana said he hoped the 31-year-old Ndlovu, who is due to start a degree in sports science after the Olympics, will inspire a new generation of black kids to take up rowing.
"We are encouraging our youth to say, "Look, let's not just focus on netball or football as the black sports,' but (that) they can get involved in any sports as long as they are given exposure at an early age."
Ndlovu took up rowing 15 years ago and has been supported well by the South African federation.
"In the past two years, I've made really important strides with the three guys standing behind me," he said, referring to his crew. "They mean a lot to me."
Britain's Peter Chambers told BBC Radio straight after the race that he was angry his boat hadn't been allocated an outside lane, which may have meant they would have been affected less by the crosswinds. His crew was the quickest from the semifinals.
"The second kilometre got a lot more fair and that's where we got our speed up," Chambers said. "FISA should be ashamed that they didn't change that in the finals."
However, Chambers said in the post-race press conference that his comments were made in the heat of the moment and that Britain had no plans to make a complaint.
Denmark, led by triple Olympic gold medallist Eskild Ebbesen, almost delivered a wire-to-wire victory after taking the initiative early on. That would have been the country's fourth victory in five Olympic finals for the lightweight four.
But the Danes faded in the final 100 metres, allowing South Africa to charge through and take a blanket finish. Denmark won the bronze, just 0.07 seconds behind Britain.
"By the last 500 metres, as the speed of the race increased, the wind was killing us," Denmark's Jacob Barsoe said. "We had lots of bad strokes and lost boat speed."
The 40-year-old Ebbesen still won his fifth Olympic medal of his career.