To help make his decision, Gardenhire channeled his predecessor, former Twins manager Tom Kelly.
Matt Capps got Bautista to pop out to third and Minnesota held on to beat the Blue Jays, 7-6.
"TK told me a long time ago, 'Don't take the ball out of the hands of your closer,'" Gardenhire said. "'Let him make the decisions. If he wants to pitch around a guy, leave it up to him.' I did it once in New York and TK yelled at me. I didn't take it out of his hands tonight."
It was Capps' sixth save in six chances.
Darin Mastroianni got his first major league hit and drove in three runs for Minnesota.
Ryan Doumit added two RBIs and Denard Span had two hits and scored twice in a game that lasted three hours 17 minutes, after rain delayed the first pitch for one hour 51 minutes. Nick Blackburn snapped a 13-start winless streak.
Mastroianni clapped his hands and yelled to himself after both of his hits. It was a rare display of excitement and jubilation in what has been a miserable season for the Twins, baseball's worst team so far.
"Just to be in the books that you got a hit in the big leagues, for it to be such an important one is something I can't imagine," said Mastroianni, who played one game with Toronto last season. "The second one was just as big as the first one."
Bautista hit two solo homers, raising his total at spacious Target Field to nine in nine career games. Minnesota stars Joe Mauer and Justin Morneau have hit seven home runs in 194 games at the three-year-old ballpark.
"It's bad news, we lost. I'm not going to make this game about me or my results," said Bautista, who slammed his bat on the ground while trotting to first after popping up in the ninth. "Bottom line, we came out here to try to win a game and we didn't. We're not going to kill ourselves over it, it's just one game."
Eric Thames and Edwin Encarnacion also homered for Toronto, which lost for only the second time in nine games at Target Field.
Mastroianni's single in the fifth scored Josh Willingham and Doumit to give the Twins a 6-2 lead.
Bautista led off the sixth with his second home run of the night and Thames followed two batters later with a two-run shot that landed in front of the black batter's eye in centre field.
After Mastroianni drove in an insurance run in the seventh with a single to left, Encarnacion went deep off reliever Glen Perkins to cut it to 7-6.
Perkins then allowed two singles, but got Colby Rasmus to ground into a double play and pinch-hitter Ben Francisco to fly out.
Willingham, not known for his defence, made a diving catch on Francisco's blooper to left field, allowing Perkins to exhale as he walked off the mound.
"I just ran as hard as these little legs could go," Willingham said. "When I got close, I knew I'd have to dive for it."
Blackburn (1-4) was pulled after Thames' home run, but left with a lead for the first time this season.
Kyle Drabek (2-4) was removed after 4 1-3 innings and 109 pitches — only 61 for strikes. He's lost four consecutive starts for the first time in his career.
Drabek's short outing also broke a 34-game streak of Toronto starters lasting at least five innings.
The Blue Jays entered with the third-highest walk total in the American League. Four pitchers walked nine Twins in the game and five of Minnesota's seven runs reach base via walk.
Minnesota failed to capitalize early, however, going 1 for 7 with runners in scoring position before getting four runs in the fifth.
When the long day was finally over early Saturday morning, Gardenhire and the Twins had something to be happy about for the first time in a long time.
"We're going to start every game at 9 o'clock and have chicken pot pie for lunch," Gardenhire said. "That's what we decided after this one."
NOTES: Gardenhire said Morneau might take some swings to test his sore wrist on Saturday. Gardenhire doesn't expect Morneau to go on a rehab assignment when he returns. ... The Blue Jays' record for consecutive games in which the starter lasted at least five innings is 43, set from May 5 to June 21 in 1996. ... Twins reliever Jared Burton pitched a scoreless seventh and has not allowed a run or hit in his last 11 1-3 innings. ... The Twins have allowed 20 homers in nine games against the Blue Jays at Target Field. ... Mauer tied his career high with three walks. ... P.J. Walters gets his first start for the Twins on Saturday. Walters was called up from triple-A Rochester after Wednesday's loss to the Angels and is 2-0 with a 7.24 ERA in two career starts. Drew Hutchinson (1-1) goes for Toronto. The rookie has allowed four or more runs in three of his four starts.