Sunderland will play Manchester City in the March 2 final at Wembley Stadium after goalkeeper Vito Mannone saved Rafael da Silva's decisive spot kick to seal a 2-1 victory in the shootout, after the semifinal finished 3-3 on aggregate after 120 minutes.
With barely a minute left of extra time, United was on the brink of going through on away goals after Jonny Evans scored a 37th-minute header to make it 2-2 on aggregate. But the match swayed Sunderland's way when United goalkeeper David de Gea fumbled in a shot from Phil Bardsley.
Javier Hernandez forced penalties, though, by scoring in the first minute of injury time.
"We did not play well enough on the night," said United manager David Moyes, who has just the Champions League title to go for in a tumultuous first season at Old Trafford.
"If we were going to (go through), we were going to stumble over the line and get through. It wasn't because we played well enough."
In a nervy shootout full of poorly taken penalties, Danny Welbeck and Phil Jones blazed shots over the bar for United and Adnan Januzaj had an effort saved by Mannone, who then dived to his right to parry away Rafael's attempt and earn Sunderland a first appearance in a major final in 22 years.
Adam Johnson had earlier missed a chance to put Sunderland through when his penalty — the team's fifth — was saved.
"Unbelievable, just unbelievable," Mannone said.
Sunderland, which is fighting relegation in the Premier League, has already beaten City this season but that was back in November. City hasn't lost in 18 games since then and will be the heavy favourite.
"We need to be practically perfect to win," Sunderland's Uruguayan manager Gus Poyet said. "I hope we aren't terrible and concede four or five goals."
While Poyet was left to savour one of Sunderland's greatest achievements, the misery continues for Moyes.
Rarely can a match in the oft-maligned League Cup have meant so much to England's biggest club — and to its under-pressure coach, who admitted this was his biggest setback so far in his turbulent reign.
Sunday's defeat to Chelsea virtually ended United's hopes of defending its Premier League title, with barely half of the campaign gone, while the team is already out of the FA Cup. The Champions League now offers the only route to silverware this season but winning Europe's top prize is unlikely on current form, even if United received a benign draw against Olympiakos in the last 16.
Behind 2-1 from the first leg two weeks ago, United didn't know whether to stick or twist after make it 2-2 on aggregate when Evans nodded home at the back post after Welbeck hooked on Januzaj's right-wing corner.
United appeared to sit back to protect the lead, inviting pressure with Welbeck withdrawn into midfield in a deeper role.
Without injured strikers Wayne Rooney and Robin van Persie as well as suspended captain Nemanja Vidic, United was unusually passive and Sunderland clawed its way back into the game.
Roared on by 9,000 boisterous fans desperate for a first Wembley Stadium visit since 1992, the visitors increased the tension inside Old Trafford — exerting pressure in the knowledge that a goal would be enough to go through.
And that pressure told when De Gea bundled a shot from Bardsley — a locally-born former United player — into his own net.
Hernandez rescued United by slamming a cross from Januzaj in off the crossbar in the first minute of injury time but the reprieve was only temporary.