Carignan, a Quebec lawyer who has been deputy government leader since 2011, was named to the Senate by Harper in 2009 after nine years as mayor of Saint-Eustache, Que.
"I am confident that Senator Carignan has the experience, dedication and respect required to increase transparency and accountability in the Senate," Harper said in a release Friday.
LeBreton announced on July 4 she was stepping down as the Conservative leader in the upper chamber.
LeBreton's decision came after months of being the main spokesperson for the government on the expenses scandal surrounding senators Patrick Brazeau, Mike Duffy and Pamela Wallin. All three were named to the Senate by Harper.
LeBreton was a member of Harper's cabinet since her appointment as Senate leader in 2006, but Harper dropped that position from his cabinet in a shuffle last month. LeBreton is due to retire from the Senate in two years.