In a posting on his Facebook page, Kent says political life doesn't focus on a cabinet seat.
Returning to the ranks of regular MPs would bring its own challenges, he says.
The former news broadcaster says he will run again in the 2015 election whether or not he is a minister.
He says there are many talented MPs in the Conservative caucus and it may be time to make room for younger blood around the cabinet table.
Kent, who will turn 70 later this month, was first elected in 2008.