Voters will go to the polls in Whitby-Oshawa in Ontario and Yellowhead in Alberta.
Both seats were held by the Tories.
The Whitby-Oshawa riding was vacated with Flaherty's death this April, while former Yellowhead MP Rob Merrifield bowed out last month to work as that province's envoy to the U.S.
Harper himself made a campaign-style stop in Whitby last week, announcing his government will be doubling the child fitness tax credit as promised.
Whoever wins the ridings will spend less than a year in office before hitting the campaign trail again, as the national voting day is Oct. 19, 2015, unless the government chooses to drop the writ sooner.
Also on HuffPost