Mancini spent four years in charge at Inter, winning two Italian Cups and three straight league titles — although one was awarded after Juventus was stripped for match-fixing.
"Our goal is to regain our rightful position as one of the very top clubs in Europe and that's why I am so happy to welcome back Roberto Mancini." Inter president Erick Thohir said in a statement.
"His track record both at Inter and elsewhere speaks for itself, and his international experience and hunger for success will I know drive the team to the next level."
Mancini, who was replaced by Jose Mourinho as Inter coach in 2008, will be presented to the media on Saturday before taking his first training session.
"An exciting new challenge, I'm happy to hug the nerazzurri fans again. See you tomorrow!" Mancini wrote on his Twitter account.
His first match back in charge will be against fierce local rival AC Milan on Nov. 23.
Inter has won only two of its past seven league matches and fallen to ninth place in Serie A.
Mazzarri joined Inter in 2013 from Napoli, which he had just led to second place in Serie A. After steering Inter to fifth and a Europa League berth in his first season, Mazzarri extended his contract by a year until 2016.
"Today I made the difficult decision" to relieve Mazzarri of his duties, Thohir said. "It was a decision that I reached in conjunction with my management team.
"It was a difficult decision because he has been continuously supportive of the direction that we have wanted to take the club, and he has worked tirelessly and selflessly for Inter, with both belief and passion. I would like to thank him sincerely for his efforts."
It is only the second coaching change in Serie A this season after Rolando Maran replaced the fired Eugenio Corini at Chievo Verona last month.
Mancini announced in March 2008 he would step down as Inter coach at the end of that season, but rescinded his decision a day later. However, he was dismissed at the end of the campaign.