The 38-year-old who spent the last four years with the Alouettes as offensive co-ordinator and assistant head coach under Marc Trestman has a game to play against his former team when Toronto faces Montreal in a CFL East Division battle on Friday night.
Both teams are 2-2, and each has won twice at home and lost twice on the road.
''The major emotions I'll have is just coming into the stadium,'' Milanovich said Thursday. ''But truthfully, once the game kicks off, there's just too much going on to even consider what's happening on the other side.
''The Alouettes organization has been great to me. I have no complaints. I'm happy where I am now. That's just how it works out.''
Milanovich has plenty of ex-Alouettes with him in Toronto, including defensive co-ordinator Chris Jones, offensive line coach Steve McAdoo and quarterbacks coach Jason Maas, as well as players like kick returner Chad Owens, tackle Chris Van Zeyl and two newly signed Argos, safety Etienne Boulay and special teams ace Walter Spencer.
But the similarity that stands out is at quarterback, where he has gone from working with future Hall of Famer Anthony Calvillo in Montreal to Ricky Ray, his chief rival of the past decade.
The veterans are 1-2 in passing yards thus far, with Calvillo leading the league with 1,316 yards on 94 completions and Ray with 1,262 on 105 passes.
''I feel very fortunate to have had a chance to coach both Anthony and this guy,'' Milanovich said. ''It's something not many people get a chance to do.''
Jim Barker, who gave up coaching duties to concentrate on being the Argos' general manager after last season, pulled off a blockbuster off-season deal to bring Ray to Toronto from the Edmonton Eskimos
''I was thrilled,'' Milanovich said. ''It was a good day in our household and I know I became a little better coach that day.
''Anthony was the first really good quarterback I coached. I learned a lot coaching him. You trust a guy with that much experience who has had success, and you have to use their knowledge and their mind. We use the same principles with Ricky. He's the guy out there doing it, so it has be something he's comfortable with or we're not going to do it.''
Milanovich was in position to become Montreal's head coach if Trestman ever got the NFL head coaching job that has eluded him for years, or landed a big NCAA gig, but the Alouettes' fifth-year field boss doesn't look to be going anywhere after signing a four-year contract extension last week. Toronto offered a chance and Milanovich grabbed it.
They have similar backgrounds, both having been offensive co-ordinators and quarterbacks coaches.
In Montreal, Trestman called the plays on offence, which Milanovich now gets to do in Toronto.
''I'm enjoying it,'' Milanovich said. ''Mentally, I didn't prepare any differently when I was in Montreal.
''We set the call sheet up as a staff, just like we do in Toronto. Unless teams change something, there's not as much thinking as you think.''
Trestman said Milanovich was ready to be a head coach.
''Scott has his own natural gifts,'' he said. ''He loves the game of football. His work ethic is as good as anyone I've been around. He gave his all here.
''I couldn't be more excited for him when it happened. I think he's going to go on to a long and very successful career because he's got the right stuff and he's aligned himself with a quarterback who will give him an opportunity to do what he wants to do. It was a matter of time. Fortunately, someone gave him a chance.''
It appears many of Milanovich's offensive ideas were shaped under Trestman, who is known for complex, detailed set-ups. Calvillo took note of that watching video of Toronto games. And he alerted Ray to it when they spoke on the phone shortly after the trade, advising him to be ready to put a lot of time into studying the playbook.
''The biggest thing is that there are so many dimensions to plays and reads, that there's no way you can just go home and not study,'' Calvillo said. ''For me, to play at this level, I had to study.
''There are certain reads we do here that they're doing there. And sometimes I rush myself on a read and I caught him doing the exact same thing.''
Calvillo is more concerned with Toronto's defence because that's what he will have to face. For all the yards and TDs Montreal has produced, the offence has gone in fits and starts through the first four games.
''This week when you have a great quarterback on the other side, the best defence is keeping him off the field,'' he said. ''So we have to sustain drives throughout the game.
The Alouettes will be without key receiver Brian Bratton, who will miss a second game with a hamstring problem. Calvillo's main target of late is tall receiver Brandon London, while last year's CFL-leading receiver Jamel Richardson looks to be bouncing back from a slow start.
Montreal's main concerns are a defence that has allowed nearly 35 points per game and special teams that have given up some long kick returns. Both will be tested by the Argonauts, who are putting up Alouette-like numbers on attack.
Chris Boyd leads the CFL in rushing with 373 yards, while Andre Durie is third in receiving with 357 and Owens is second in combined yards with 1,010 behind another former Alouette, Calgary's Larry Taylor with 1,046.
The Alouettes released veteran defensive tackle Aaron Hunt, their top free agent signing, this week to give Alan-Michael Cash a start. They win also bring lineman Ventrell Jenkins back after sitting out a week.