The Calgary-based railway (TSX:CP) says the appointment is effective as of Friday.
During his time in government, Prentice was in charge of the Environment, Industry and Indian Affairs and Northern Development portfolios.
He's currently senior executive vice-president and vice chairman at the Canadian Imperial Bank of Commerce (TSX:CM) and serves on the boards of Bell Canada (TSX:BCE) and Coril Holdings Ltd.
"I am proud to be part of this great Canadian company," said Prentice in a release.
"The CPR is a foundational part of Canada' economic infrastructure and it is an exciting time to join the board as the company moves forward. I look forward to working with the board as we guide the company on its transformational journey."
Just over a year ago, a bruising proxy fight with the railway's largest shareholder culminated in the departure of Canadian Pacific's then-CEO, Fred Green, along with several board members.
Pershing Square Capital Management, led by Bill Ackman, campaigned to replace Green with Hunter Harrison, the retired chief executive of CP rival Canadian National Railway Co. (TSX:CNR).
Since taking the helm of CP last summer, Harrison has implemented a number of cost-cutting measures, including staff reductions and moving the company's headquarters out of downtown Calgary.