Paulinho, Etienne Capoue and Vlad Chiriches scored their first goals of the season before Danny Rose completed Tottenham's remarkable turnaround by the 52nd minute.
"The most important thing is how the team came back into the game and showed character to go to the next round," Tottenham manager Mauricio Pochettino said.
It was a tighter game for surprise Premier League highflier Southampton, which edged past second-tier club Ipswich 1-0 through Shane Long's 19th-minute strike.
Southampton, which is third in the league, will host Crystal Palace in the fourth round on Jan. 24, when Tottenham plays Leicester at White Hart Lane.
Bradford's reward for routing 10-man Millwall 4-0 in another replay on Wednesday is a trip to Premier League leader Chelsea after the third-tier team progressed from the third round for the first time in 15 years.
It was a frenzied opening 10 minutes at White Hart Lane where slack defending by Tottenham's much-changed team proved costly.
Martin Sordell volleyed past backup goalkeeper Michel Vorm after connecting with a long ball inside three minutes. Ross Wallace's free kick then deflected off Tottenham striker Roberto Soldado, wrongfooting Vorm five minutes later, as Burnley raced into an unlikely 2-0 lead.
But Tottenham responded quickly, with Andros Townsend's cross headed on by Soldado for Paulinho to stab into the net in the 10th minute.
There was a dreadful moment just before halftime for Soldado when the striker hit the crossbar after Townsend delivered the ball to his feet just five yards (meters) from goal.
"In football you don't need to apologize, because we are sorry for him because I think he deserved a goal," Pochettino said.
But the hosts did go in at the break level after Capoue scored only his second Tottenham goal in his second season at the London club. Capoue drove the ball into the net from the edge of the penalty area from Ben Davies' cutback.
The start to the second half was as frenzied as the opening of the game, with Chiriches heading in Paulinho's corner at the far post after four minutes to put Tottenham in front.
The turnaround was completed inside two minutes when Benjamin Stambouli's high crossfield pass reached Soldado, who sent it across the goal for Rose to net Tottenham's fourth.
After six goals, the remainder of the game couldn't match up to the opening 52 minutes.
"When you are 2-0 up and you make the mistakes we made it is going to be very tough," Burnley manager Sean Dyche said. "We made the errors."