Jon Stead scored for a fifth consecutive round in the second half after an own goal from John O'Shea gifted Bradford the lead in the third minute.
Bradford, which beat Premier League leader Chelsea to reach the fifth round, is now only one win from a Wembley Stadium semifinal.
"There was always a danger that because we beat Chelsea everyone would expect us to win today," Bradford manager Phil Parkinson said. "We had to guard against that and go in there with the underdog mentality.
"Right from the first whistle we played with great desire and great spirit and we were first to every ball all over the pitch. Maybe we took Sunderland by surprise."
Bradford, which was relegated from the Premier League in 2001 and is currently ninth in League One, produced another stunning cup run in 2013 when it reached a League Cup final won by Swansea.
"We have done it again and we are obviously buzzing," Stead told BBC radio. "It was a terrific start for us to get the goal and we just carried on from there."
Sunderland manager Gus Poyet maintained that the loss "is not humiliating."
"If the top teams always beat the smaller teams, then the cup would not exist," Poyet said.
It was a routine day, though, for Arsenal. Middlesbrough, which is second in the second tier, never looked like emulating its fourth-round victory over Premier League champion Manchester City.
Olivier Giroud scored both of Arsenal's goals in the 2-0 win to take his tally in the season to 10.
The France forward turned in Kieran Gibbs' cross in the 27th minute and netted again inside two minutes after meeting Alexis Sanchez's corner at the near post.
"Giroud is today a different player to the guy who arrived here (in 2012)," Arsenal manager Arsene Wenger said. "He has improved tremendously his mobility, his technical quality."
The closest Middlesbrough came was in stoppage time when Kike headed against the post.
The day's all-Premier League encounter was closer, with Aston Villa beating Leicester 2-1.
Leandro Bacuna broke the deadlock in the 68th minute, and Scott Sinclair doubled Villa's lead in the 89th, with goalkeeper Mark Schwarzer helping the ball into the net.
Andrej Kramaric headed one back for Leicester a minute later, but Villa held on with new manager Tim Sherwood watching from the stands. The former Tottenham manager delivered an impromptu halftime team talk despite leaving coach Scott Marshall in charge for this game.
"He came in and made a couple of points to the lads," Marshall said. "I thought the boys delivered that in the second half and were very good."
The fifth round will be completed on Monday when Manchester United plays at Preston.
Blackburn, Liverpool, Reading and West Bromwich Albion won on Saturday to secure a place in Monday's draw for the last eight.