The former sevens captain racked up a hat trick on the stroke of halftime, giving Fiji a 32-15 lead, then claimed the fourth in the 52nd minute of an exciting contest before 10,000 fans.
"This is the first time I've scored four tries," he said. "It's an honour and a privilege."
In ideal spring weather at Rotorua International Stadium, Goneva became the first Fijian to score four in a World Cup match, and the first in a test since Rupeni Caucaunibuca against Chile in 2003.
"Some fantastic efforts, especially by Vereniki — that was pretty special on debut," captain Deacon Manu said. "It was just fantastic to get the result.
"We probably didn't play in the right areas but when we did get out of our own half we put some pressure on. To the credit of the Namibians, they put pressure on the whole match any way they could."
In another tougher-than-expected display by a rank outsider, Namibia showed just as much skill and joy at running the ball as the Fijians, scoring two classy tries, dominating long periods, and deservedly finishing with its most points in a World Cup match. It was also debuting two starters.
However, the loss extended Namibia's winless record at the cup to a dozen matches since 1999.
Namibia, expected to receive a defeat on the scale of its only previous match with Fiji, 67-18 in 1999, gave the game a great start when flyhalf Theuns Kotze elected to use the tailwind to kick a penalty from 56 metres, and made it.
Fiji replied immediately. Namibia fullback Chrysander Botha lost the ball in a ruck deep in his half, and quick hands gave Fiji a 3-on-1 with Goneva in the right place for the try.
Seremaia Bai, the only starter remaining from Fiji's last World Cup appearance in the 2007 quarterfinals, kicked over the conversion, the first of his 19 points from eight kicks at goal. He missed just one.
Undaunted, Namibia forced Fiji back into their half three straight times and Kotze nailed three successive dropped goals in four minutes, the cheers growing louder each time for his and Namibia's audacity.
Namibia jumped to a 12-7 lead, then it was Fiji's turn to roll. Bai cut the deficit with his first penalty, and from the restart, flyhalf Waisea Luveniyali attacked the left side and kicked ahead. Its defence in tatters, Namibia wasn't anywhere near lock Leone Nakarawa when he strolled over.
Fiji pounced again from the restart when Namibia No. 8 Jacques Nieuwenhuis dropped an easy catch. The ball bounced to Nakarawa, and the former soldier who left the army to make the squad, stepped inside and passed out to Goneva for an easy run in from 40 metres out. In the space of four minutes, Fiji scored 15 points and led by 10.
Bai's second penalty made it 25-12 in the 27th, then Namibia showed how comfortable it was giving the ball air, with Kotze dictating the attack.
Kotze tapped a penalty instead of kicking for goal, set up switches and cut-out passes, and Namibia impressed. Twice, Namibia came close, but centre Danie van Wyk was dragged back just when he appeared to have beaten the last defender, and couldn't control a Kotze chip behind the tryline.
"We were surprised because only Fijians play open rugby, but they played open rugby, too," Nakarawa said.
Just before halftime, Fiji produced a try from nowhere, when it turned over ruck ball outside the Namibia quarterline, went the shortside and Goneva had only a prop to beat. The conversion made it 32-15.
The action didn't let up after the break. Namibia inside centre Piet van Zyl dummied through a gap, was backed up by winger Conrad Marius, and lock Heinz Koll angled to the line.
Goneva crossed for his fourth after centre Gaby Lovobalavu went close, then Namibia showed its pluck with a setpiece try finished by fullback Botha to make it 39-25.
"We said at halftime to keep the ball in hand. We looked dangerous, ball in hand," Namibia captain Jacques Burger said. "Unfortunately we gave it away too many times and when we kept it we looked dangerous. So we've got to keep the ball."
When Bai knocked over a late penalty, it looked like he wanted to give his teammates a rest, but they struck just before the end from a lineout drive, and the ball was delivered to the short side where winger Naipolioni Nalaga was too fast and strong from 10 metres out. Nalaga matched his father, Kavekini, who also scored a World Cup try against Argentina in 1987.
Fiji move on to Wellington to face South Africa in a week, while Namibia stay in Rotorua to meet Samoa on Wednesday.