Sweeney is the team's eighth GM after serving as an assistant for the past six years and working three before that in player development and hockey operations.
The Bruins missed the playoffs this season after seven consecutive trips, and that failure cost Chiarelli his job.
Sweeney, 48, inherits a veteran, cap-strapped roster and coach Claude Julien, whose future remains unclear.
After winning the Stanley Cup in 2011 and going to the final in 2013 with the same core that remains, Boston expects success, and Sweeney knows it.
"I am fully aware of everyone’s expectations to move the organization forward," Sweeney said in a statement. "The challenges ahead rests with the players, the coaches and the management group to work hard to make the necessary changes to bring the Bruins back to the forefront of contending for the Stanley Cup."
CEO Charlie Jacobs and team president Cam Neely will introduce Sweeney at an afternoon news conference at TD Garden.
With Sweeney hired by the Bruins, the only other current GM vacancy in the NHL belongs to the Toronto Maple Leafs.