The ugly incident occurred shortly after the half-hour mark, after Fiorentina had fallen behind 2-0.
As he walked toward the bench, Ljajic put his thumb up to sarcastically applaud Rossi for the substitution. When the player sat down, Rossi ran over and appeared to slap Ljajic in the face.
As it appeared Rossi was lining up a punch, the coach was pulled away by his assistants and other players.
Fiorentina president Andrea Della Valle announced Rossi's firing after the match, without immediately announcing a replacement.
"I spoke with Delio Rossi and he's prepared to apologize, but this was a choice that had to be made," Della Valle said on Italy's Sky TV. "It's a choice I should never have had to make, but there is no justification for Delio Rossi's actions. The firing is for his well being. He's a great person."
Della Valle indicated that a replacement would be announced Thursday.
Fiorentina is having a miserable season. Rossi replaced the fired Sinisa Mihajlovic in November but has been unable to turn the Tuscan club's season around. Fiorentina is tied for 14th place with two rounds remaining in Serie A.
"I can understand the tension of the moment but losing your cool in that manner is inexcusable," Italian Players' Association president Damiano Tommasi said, according to the ANSA news agency. "It's also strange that it came from Delio Rossi, who is always very well balanced in what he says."
The firing marked the 19th coaching change in the Italian league this season. The previous Serie A record for changes was 15 in 1951-52. Ten clubs have changed managers at least once, with Cagliari leading the way with three firings.