CN issued a statement Thursday night saying it was notified by Teamsters Canada, which represents conductors, yard workers and other train workers, that its members rejected the deal.
The latest tentative deal was reached in early February in the face of a threatened strike by the employees.
It came as Labour Minister Kellie Leitch indicated that the federal government was preparing back-to-work legislation.
Leitch had expressed concern the strike would cause disruption to Canada's economy.
The workers had rejected another tentative contract last fall.
Claude Mongeau, CN's president and chief executive officer, expressed disappointment at the workers' decision.
"CN today tabled an offer to the (union) to settle all unresolved contract issues through final binding arbitration to allow the parties to move forward without the prospect of labour disruption," Mongeau said in a statement.
"A labour dispute now would ill serve CN's customers, the Canadian economy or the company's employees."
The union did not immediately comment on Thursday's developments.
CN says it transports about $250 billion worth of goods annually for a wide range of business, including natural resources and consumer goods.
Also on HuffPost