Toronto's Daniel Nestor and Frank Dancevic of Niagara Falls, Ont., fell to Japan's Kei Nishikori and Yasutaka Uchiyama 6-3, 7-6 (3), 4-6, 6-4.
Both teams made player substitutions prior to the match as Dancevic went in for Vancouver's Vasek Pospisil, who is still recovering from a back injury. Nishikori subbed in for Yuichi Sugita on the Japanese side.
The match marked the first time Dancevic and Nestor had played together in three years.
"I thought those guys came out really strong and we really didn't take advantage of our chances," said Canadian captain Martin Laurendeau. "The guys fought really hard and it was really a matter of a few points."
The Canadian pair got out to an early lead in the second set but Japan broke back to force the tiebreak, and capitalized to take a two-set advantage before Nestor and Dancevic captured the third set.
For Nestor, the loss was only his eighth in Davis Cup doubles play in his entire 22-year career representing Canada. It was also the first loss for Dancevic and Nestor as a pair, who entered the contest undefeated together with a 3-0 record.
Canada is currently ranked No. 7 on the Davis Cup Nations ranking while Japan sits at No. 12. Japan owns a 5-0 record against Canada in Davis Cup action, but the two countries had not played each other since 1938 prior to this tournament.
Dancevic will play World No. 18 Nishikori in a must-win singles match on Sunday. If a fifth and final match is needed, Peter Polansky of Thornhill, Ont., will face Go Soeda, also on Sunday.
The winner will advance to the quarterfinals to face either the Czech Republic or the Netherlands.
The nations split their opening pair of first-round singles matches on Friday, with Nishikori defeating Polansky in straight sets 6-4, 6-4, 6-4, before Canada pulled even on the strength of Dancevic's 6-4, 7-6 (2), 6-1 victory over Soeda.