Japan forward Yuki Otsu broke away from his marker during a corner in the 34th minute and the ball landed right at his feet for a simple tap-in past Manchester United goalkeeper David De Gea. Spain, reduced to 10 men a few minutes after the goal, never managed to get back into the game after that.
"It was an important win for us," captain Maya Yoshida said. "Spain are a very strong team in the group and we played well against them. We worked really hard and we will enjoy our victory. We can get a lot of confidence from this."
The quick and creative Japanese took full advantage of the extra man, unleashing wave after wave of attack.
"It feels good to have beaten Spain," Japan forward Kensuke Nagai said. "We have never beaten them before at any level. We look forward to now trying to get to the next stage (of the tournament)."
The loss was a huge blow for Spain's under-23 team, who wants to win Olympic gold to add to the World Cup and European Championship titles held by the country's full national team.
But the defeat wasn't insurmountable.
Spain, touted as one of the favourites to win a gold medal, will now look to the national side's experience at the 2010 World Cup for inspiration. Spain lost its first group stage match in South Africa to Switzerland, but went on to win the tournament.
The Olympic side has two more matches to turn it around, against Group D opponents Honduras and Morocco.
"Our aim is to win both of them. We have to," forward Juan Mata said. "We need to improve some things and we have to play better."
Spain started Thursday's match brightly with good build-up play and quick passing in the first 20 minutes, but they couldn't break through a solid Japan defence and didn't have a meaningful shot on goal.
It wasn't only Japan's back line that was troubling the Spaniards. Nagai caused major problems for them throughout the match, slipping behind defenders with quick runs. Spain centre back Inigo Martinez was sent off for a foul while trying to stop Nagai from scoring. The forward continued to badger the defence, shaving the post in the 58th minute.
Spain's youngsters mimicked the slick passing game the national team is so well known for, pressuring Japan but failing to test the goalkeeper with any dangerous shots. Japan's defenders were also able to strip the ball off Spain's attackers several times in the penalty area as they threatened to shoot.
Japan has been rising steadily in world football, qualifying for the 2010 World Cup and building a strong youth program. The victory against Spain rates among Japan's greatest.
But coach Takashi Sekizuka was modest when asked if this was the case.
"We looked forward to this match very much, and showing how much we have was the main point," he said. "We are very pleased with the result."
For Spain coach Luis Milla, the turning point in Thursday's match was Martinez's red card. He said the tables were turned on the Spaniards, and it was they who were chasing the game.
"With one player less, we were looking for an equalizer, which isn't our style," Milla said.