Jean, 52, is the prime suspect in the murder of Juthlande Pierre, 42, whose body was found in her Roxboro home on Jan. 15, 1996.
Jean was on the run until earlier this month, when authorities in New Jersey located him and notified Montreal police.
Police in New Jersey arrested the man on Monday, and Jean is expected to appear in a New Jersey court on Tuesday.
Montreal police say the last time Pierre was seen alive was on Christmas Day in 1995, at a party with Jean. After friends of the woman filed a missing persons' report several days later, police found Jean's body hidden in her home with multiple stab wounds to the head.
Police put out an arrest warrant for Jean and an Interpol red alert, but for years they could find no trace of the man.
It's not known how US authorities located Jean, but police say they were able to confirm his identity through fingerprints.
He is likely to be extradited to Canada within a short time, to face charges in Montreal related to Pierre's death — the fourth murder of 1996.