Bouchard, making her debut in the Tokyo tournament, broke Jankovic to go up 5-2 in the second set and then held serve to win the match in 1 hour, 10 minutes at Ariake Colosseum.
Jankovic had a 5-4 lead in the first set but Bouchard won the last three games after a talk with her coach.
"I felt like I wasn't playing my best in the first set," Bouchard said. "But I stayed with her and started to get a lot more aggressive near the end of the first set and that was the turning point."
Bouchard, who is 46th in the world rankings, said winning last year's junior Wimbledon title has given her confidence to compete at a higher level.
"That gave me a lot of confidence to play in the pros," Bouchard said. "Playing week in, week out against the top players has made me a better player."
The 19-year-old Bouchard will next face either Venus Williams or Simona Halep of Romania, who play later Wednesday.
In another third-round match, fourth-seeded Caroline Wozniacki of Denmark, defeated Magdalena Rybarikova of Slovakia, 6-1, 6-1.