The ballot took place on Wednesday three years and two days after Gillard ousted Rudd in a similar internal government showdown.
It makes him leader of the party, but not prime minister.
Party official Chris Hayes says Gillard lost 57 votes to 45.
Rudd will likely have to demonstrate that he can command a majority of lawmakers in the House of Representatives before the governor-general makes him prime minister.
If he cannot, opposition leader Tony Abbott could be asked to form a government or the elections could be moved up from September to August.
Also on HuffPost