Milanovich said his daughters Macall, 11, and Maggie, nine, initially cried then he told them he was assuming the head job in Toronto. It wasn't so much having to move to a new city after five years in Montreal, but rather the idea of sitting in the stands and having to listen to Argos fans criticize the head coach during the roller-coaster ride that usually is the 18-game CFL regular season.
But the 40-year-old native of Butler, Pa., often pressed the right buttons as a rookie head coach, leading Toronto to its first Grey Cup title since 2004. On Thursday, that earned Milanovich the CFL's coach of the year award.
However, Milanovich's highlight this season was celebrating Toronto's 35-22 win over the Calgary Stampeders in the 100th Grey Cup on the Rogers Centre field afterwards with his family.
"The moment after the Grey Cup was one of the best of my life, spending it with my wife, my daughters and my family," Milanovich said. "But the thing people don't see is after the losses, the concern on their faces.
"They're hurting because you're hurting and that's as much of a motivator for me as anything. What the typical outside world doesn't realize is coaches are paid to do that but the families live and die with every win and loss so for them to be able to share that moment with me on the field and be in the parade and celebrate in our city, those are the reasons you do it, for those moments."
Milanovich claimed the Annis Stukus Trophy at a luncheon in Regina after receiving 34 of 45 first-place ballots in voting by the Football Reporters of Canada. Calgary head coach/GM John Hufnagel and Mike Benevides of the B.C. Lions were the other finalists.
Milanovich became the sixth Argos coach to capture the award and second in three years as GM Jim Barker won it in 2010.
Barker stepped down as Argos coach in December 2011 and hired Milanovich, who had served as an assistant in Montreal for five seasons. Barker then made headlines when he landed veteran quarterback Ricky Ray in a blockbuster trade with the Edmonton Eskimos.
But Ray and the Argos started slowly. In August, Toronto released running back Cory Boyd even though he led the CFL in rushing at the time and the team was just 4-4 heading into its annual Labour Day showdown with Hamilton.
After consecutive wins over Hamilton, Toronto then lost five of its next six games, including two-of-three when Ray went down with a knee injury, to stand at 7-9. However, Ray and the Argos caught fire, winning their final five games and capping their late-season run with a Grey Cup victory before a rabid Rogers Centre gathering of 53,208.
"It was a great year but sometimes it's even sweeter when you have to overcome some obstacles," Milanovich said. "It doesn't always go smoothly but the roll we got on at the end of the season is the kind of roll you dream about . . . to watch that happen in our stadium, in our city, that's a moment I'll never forget."
Hufnagel, the CFL's top coach in 2008, did a great job of leading Calgary to its second Grey Cup berth in five years. The Stampeders finished second in the West Division with a 12-6 record despite losing starter Drew Tate for most of the season.
Benevides, a Toronto native, served as the defensive co-ordinator on B.C.'s 2011 Grey Cup-winning team before replacing Wally Buono as head coach. Benevides led the Lions to a league-best 13-5 record, the 13 victories tying a club record for the most by a first-year head coach.
But Calgary spoiled B.C.'s quest for a second straight Grey Cup crown with a 34-29 win in the West final.
A successful title defence is the challenge now facing Milanovich, although he was Montreal's offensive co-ordinator when it won successive CFL titles in 2009 and 2010.
And with Milanovich and Ray both having the benefit of a year together under their belt, Milanovich said Toronto should be able to start 2013 stronger than it did last year.
"When that's the case and you continue to get better throughout the season as we did last year, then it stands to reason at the end of the season you're even better than you were the year before," Milanovich said. "But there's a lot of work that has to be done before any of that happens."
However, Toronto will have the benefit of continuity as both Milanovich and Barker have signed contract extensions with the CFL club.
"Hopefully we in Toronto can start to get that same continuity these other winning organizations have had and build upon the success we had this year," he said. "Jim laid the foundation for what happened this year and not only that, he had the courage once he hired me to let me do my job my way.
"I know that had to be hard for him because he's been a coach for many many years and as close as we are we don't agree on everything. But he stepped back and let me make my mistakes and let me learn on my own . . . I'm really thankful for his friendship and leadership and all he's meant to my career."