In a six-to-one decision, a majority of justices on the top court ruled that Nadon doesn't qualify to join them on the court.
"Today's judgment will be of great importance, especially in constitutional matters," said Sébastien Grammond, who represented two groups of retired Quebec judges in the case. "First and foremost because it makes important statements as to how the Constitution of Canada can be amended."
Toronto lawyer Rocco Galati had challenged Harper's appointment of Nadon. Galati argued he didn't qualify for one of the court's three Quebec seats, because he came from the Federal Court and not from a Quebec court.
Quebec judges are a special category because of the province's unique civil legal code, different from the common-law code in the rest of the country.
Harper referred the question to the court and Nadon stepped aside until the matter could be decided.
The decision may also provide insight into how the court will rule on the Senate reference, Grammond suggested, which includes the question of whether Parliament can reform the Senate without reopening the Constitution.
"It will take months and years perhaps to understand" all the implications of the decision, he said.
Mobile users, click here to watch live as experts react from the Supreme Court.