But after Sunday's loss to the Denver Nuggets, he probably wishes they could.
The Nuggets reserves came off the bench to outscore their Raptors counterparts 72-16 as Denver pulled away in the fourth quarter for a comfortable 112-98 win before 16,290 fans at Air Canada Centre.
"Our bench has got to give us something," Casey told reporters after Toronto (6-10) lost its third straight. "Our guys (starters) can't play for 48 (minutes), so we've got to get production from our bench.
"The guys coming in have got to develop a toughness, a resilience of getting stops. And, it starts on the defensive end. In the second half, they shot it well but we didn't make them feel us."
The 72 bench points conceded is the worst in Raptors' history, exceeding the 69 points surrendered to the Chicago Bulls in a loss on Jan. 25, 2006.
Diminutive guard Nate Robinson led the way for Denver (10-6) with 23 points off the bench. The 5-foot-9 veteran hit five three-pointers on the afternoon, including back-to-back shots from beyond the arc with just over two minutes to go, as the Nuggets coasted to their sixth straight win.
In addition to Robinson's team-high 23 points, three other Denver players had double-digit point totals off the bench, including Timofey Mozgov with 16, Darrell Arthur with 14 and Jordan Hamilton with 10.
Rudy Gay led the way for the Raptors with 23 points. Jonas Valanciunas had 18 points and 11 rebounds.
Finishing 1-3 on their longest home stand of the season, the Raptors now head out West for the next week, visiting the Golden State Warriors, Phoenix Suns and Los Angeles Lakers.
"Personally, I'm frustrated. Period," said Toronto guard DeMar DeRozan, who finished with 17 points. "We've just got to figure it out and turn it around on the road.
"It's correctable but we shouldn't have to keep doing the same thing over and over before we realize that we are doing something wrong. We have to understand what we're doing out the gate and correct it."
Casey shuffled his line-up, inserting Tyler Hansbrough into the starting rotation in place of a struggling Amir Johnson. And, the initial results were positive, as the Raptors came out flying, opening up a 15-point lead eight minutes into the first quarter on the strength of a swarming defence and some deadly accurate field goal shooting.
DeRozan hit his first four shots and Lowry sunk his initial two attempts from beyond the three-point arc.
But the Nuggets went on a 13-4 run over the final 3:56 of the first quarter to cut the lead to 31-25.
Denver led 47-45 at the half and had a 76-72 lead after a see-saw third quarter that featured four lead changes.
In addition to lamenting his bench being outplayed, Casey also said the Raptors didn't pass the ball enough.
"We had only 18 assists, they had 29," he said. "And, that's where it starts in the offensive end.
"We talked about the defensive end but the offensive end, we've got to start trusting the pass."
Gay said despite three straight losses, the Raptors are not getting down on themselves.
"If we want to win, we have to make the right adjustments," he said.
"Let's not get crazy," Gay said when asked if there might need to be some dramatic shake-up with the team. "They hit some tough shots and we made them hit some tough shots. We made them move the ball around, which is our defensive principle. We can't look at ourselves like we are just terrible.
"We made some strides today but we can be better."