Williams defeated the Canadian 6-3, 6-7, 6-3, but she acknowledged that Bouchard made it difficult for her.
"She played so well," Williams said. "She's going to be a great player — she's already a great player — and it was just a fight to the end."
Bouchard called Williams a "legend."
"I watched her when I was younger," she said on the WTA's website. "She's a legend and a really great player. So I was really excited to play against her — but I didn't take my chances enough, and against a player like Venus you need to."
The 19-year-old Bouchard put Williams on notice when she built a 3-1 lead. Williams fought back to break the teenager three times in the first meeting of their careers.
In the second set, Bouchard capitalized on her second set point, winning the tiebreak 7-4.
"I was just fighting until the end," Williams said. "It was disappointing to lose the second set, but to win the third set is the goal, so I'm happy."
Bouchard had upset sixth-seeded Jelena Jankovic of Serbia 7-5, 6-2 on Wednesday, a day after ousting No. 8 Sloane Stephens of the U.S.
"I'm always disappointed to lose — I played some good matches here, but I always want to do better," Bouchard said. "I don't think I played as well as I can, but obviously Venus is a great player and she put a lot of pressure on me throughout the whole match. I never really felt in control on the court."
Bouchard, making her debut in the Tokyo tournament, is 46th in the world rankings.
Williams will face Petra Kvitova in the semis. The seventh-seeded Czech beat Svetlana Kuznetsova 6-4, 6-1.
Also, fourth-seeded Caroline Wozniacki beat Lucie Safarova of the Czech Republic 2-6, 6-3, 6-2 to set up a semifinal with fifth-seeded Angelique Kerber of Germany, who ousted second-seeded Agnieszka Radwanska of Poland 6-4, 6-4. Wozniacki and Kerber are 3-3 head to head. Wozniacki won their semifinal at Indian Wells in March.
— With files from The Associated Press