In a fitting way to mark United manager Alex Ferguson's 1,000th league game in charge, Van Persie equalized in the 87th minute before heading home a corner in the second minute of injury time for his fourth goal since joining from Arsenal for 24 million pounds ($38 million) in the off-season.
Arsenal showed earlier there is life after Van Persie when new signings Lukas Podolski and Santi Cazorla scored their first goals for the club in a 2-0 win over struggling Liverpool.
The loss leaves Liverpool rooted in the bottom three after the club's worst start to a season in 50 years, compounding its woes after a failure to add some much-needed signings to the squad on Friday's transfer deadline day.
In the day's other match, Aston Villa earned its first point of the campaign with a 1-1 draw at Aston Villa, only being denied victory because of Hatem Ben Arfa's stunning second-half equalizer.
Van Persie's late show, which took him to 100 Premier League goals in 197 games, will have come as a mighty relief for the Netherlands striker, after his impudent chipped penalty was clawed away by goalkeeper Kelvin Davis in the 68th minute.
The score was 2-1 to Southampton at the time and United looked headed for a second loss in three games before Van Persie's last-gasp intervention.
"I don't know what I was thinking with the penalty. I was going to hit it hard but at the last second I changed my mind," Van Persie said. "I'm quite disappointed about it. I ask certain standards from my game and at 2-1 down you cannot take a penalty like that."
Ferguson, famous for what the British media calls the "hairdryer treatment" in the dressing room when his team or player underperforms, was all smiles after the match despite Van Persie's error of judgment.
"I was surprised because every time I've seen him take a penalty he usually rattles it into the corners, so I was confident," Ferguson said. "But he made up for it."
Having started the season with a 1-0 loss at Everton, this win lifted United up to fifth place on six points, one behind neighbour and reigning champion Manchester City and three behind early leader Chelsea.
Arsenal is in eighth place, a point further back than United.
United's defence looked shaky throughout at St. Mary's Stadium, with both of promoted Southampton's goals stemming from crosses to the back post where United's full backs were outwitted.
Rickie Lambert put the hosts in front in the 16th, beating Rafael da Silva in the air and heading home a trademark finish.
Van Persie equalized when he chested down a cross from Antonio Valencia and volleyed into the corner. But Southampton regained the lead in the 56th through Morgan Schneiderlin, beating a slipping Patrice Evra to a cross and planting a header into the corner.
"We have to take positives from our performance," said Southampton manager Nigel Adkins, whose side is the only one in the Premier League without a point. "We are scoring goals and what we have to do is stop conceding them at the other end."
The game changed when Ferguson sent on Paul Scholes in the 61st, with Southampton suddenly backing off and allowing the veteran midfielder to control play.
The Liverpool-Arsenal match was already being seen as a must-win game for both teams, and it was the visitors who sealed their first win and goals of the season with an encouraging display at Anfield.
Podolski, the Germany forward, and Cazorla, the Spain midfielder, set each other up for their goals, while fit-again midfielder Abou Diaby was the standout player on the pitch and Arsenal also kept a third straight clean sheet.
"The sharpness gets better from game to game and we looked solid and scored two good goals," Arsenal manager Wenger said. "''Everybody understood quickly Cazorla would not need six months to settle. Lukas Podolski is deadly when he gets a chance. I am pleased for them."
Liverpool lacked a cutting edge up front, and failing to sign a striker this week after loaning out Andy Carroll to West Ham could come back to haunt manager Brendan Rodgers.
"I am not here to cry," Rodgers said. "I felt we needed reinforcements — that is the reality. But it is gone now. The window is shut."
Ciaran Clark put Villa ahead at St. James' Park before Ben Arfa equalized with a rising drive from 20 yards (meters), rescuing Newcastle after a poor home performance.