Bradley Wright-Phillips took the penalty kicks for the Red Bulls (4-5-5) in the fifth and the 65th minutes — and both went waist high and to Ousted's right.
"I think the first one (stop) was really important," Ousted said. "They came out guns blazing and that would have capped a good first 10 minutes for them. So, important to save that and keep us in the game and then right after they get the red card and we go up and score so that was definitely important.
New York's Sacha Kljestan picked up the red card in the 11th when he elbowed and tripped Kendall Waston.
Kekuta Manneh opened the scoring with a close-range goal in the 15th minute, and Kianz Froese added another for the Whitecaps (9-6-2) with a breakaway in the 77th.
Anatole Abang pulled New York back within one in the 82nd, but Ousted made his most difficult save of the match a minute later, diving and deflecting Manolo Sanchez's low-skipping shot out of play.
"He's been outstanding but up to this point he hasn't saved too many penalties," Vancouver coach Carl Robinson said of Ousted's play.
Vancouver moved into a first-place tie with Seattle in the Western Conference.
The Red Bulls lost their fourth league match in a row.
"That was a bizarre game, I think that's a fair word to explain it but we came out on the right end of a game that had a little bit of everything," Robinson said.
"They're a good team, they play a certain way and we were lucky we got the three points tonight."
With files from The Canadian Press