The result means Japan finished second in Group F and will play either hosts Britain or Brazil in Cardiff on Friday. If it had won the match, it would have faced the United States, the top-ranked team in the world, or France in Glasgow, also on Friday.
Japan's coach, Norio Sasaki, said he made tactical substitutions in the second half and gave the players instructions to keep possession rather than trying to attack, based on how the other match in Group F was going. In that match, Canada drew 2-2 with Sweden, meaning Sweden topped the group.
"It was a different way of playing compared to the usual game, but the players were on the same page as me," Sasaki said through a translator.
He said introduced one striker, Yuki Ogimi, late in the match, so that "we could take one goal back" if South Africa — which lost its first two games — managed to score.
"I feel sorry we couldn't show a respectable game, but it's my responsibility, not the players, why the game was like that," he said. "It was important for us not to move to Glasgow."
Having already secured a quarterfinal spot, Sasaki began the match with just four regular starters.
In the 18th minute, Japan captain Kouze Ando had a good chance, but sent a header wide.
The team appeared to show more bite after veteran attacking midfielder Nahomi Kawasumi was brought in the second half.
Ando had another chance to score just past the hour mark but missed the target from close range. Minutes later, Asuna Tanaka headed over the bar from a decent position.
Having beaten the United States last year in Germany to win the World Cup, Japan is now seeking to win its first Olympic medal in three consecutive attempts. South Africa's young side, all but one of whom are students, failed to create any clear-cut chances. It had already been eliminated from the competition.
The match at Cardiff's Millennium Stadium was played under a roof, the first time that has happened in Olympic football.