Rick Nash scored for the fifth straight game, and the New York Rangers used a four-goal second period to take the lead for good.
The Devils led 1-0 and 3-1 in the first, but New York got within one before intermission and then swarmed future Hall of Fame goalie Martin Brodeur.
After Dominic Moore and Marc Staal had goals in the first for the Rangers, Mats Zuccarello scored two straight to put New York ahead for the first time. Carl Hagelin and Nash found the net, too, behind the beleaguered Brodeur, who angrily swatted the puck away after one of the tallies.
Nash has seven goals in his streak and 18 this season.
A day after the Los Angeles Kings and Anaheim Ducks played in 60-degree weather at Dodger Stadium, the NHL returned to conditions more fitting for hockey.
Lundqvist, who sported stylish pinstriped pads for the occasion, stumbled through a difficult first but settled down and made 19 saves.
He hadn't allowed more than two goals in his previous seven outings. The Devils took care of that in the first.
Jaromir Jagr had two assists to excite the large number of New Jersey fans who made the trek to the Bronx for what was nominally a Devils home game.
The teams played through the second period as the snow started and increased in intensity, not long after they were forced to wait around an extra hour to drop the pick because of bright sunshine that created a dangerous glare on the ice.
The temperature was 24.9 degrees when the first puck dropped at 1:41 p.m. in front of a sellout crowd of 50,105.
Yankee Stadium's second hockey game will take place on Wednesday night when the Rangers will again be the road team in a matchup with the New York Islanders.
Zuccarello tied it 3-3 at 2:48 of the second when he redirected in a pass from John Moore, and put New York in front at the end of a 3-on-1 rush with 7:16 left. Benoit Pouliot chipped the puck ahead at centre ice to Derick Brassard, who sent a pass from the right side to Zuccarello for his 15th goal.
Hagelin made it 5-3 just 1:09 later when he flipped a shot from the right circle that appeared to hit Devils defenceman Marek Zidlicky before getting through Brad Richards' screen and by Brodeur.
The Devils netminder was again victimized by an odd-man attack that Nash finished with 28.5 seconds left in the middle frame. Brodeur was serenaded with derisive chants of "Mar-ty ... Mar-ty" as Rangers fans in the stands made their voices heard.
He watched the rest of the game from the bench, as Cory Schneider replaced him to start the third. Brodeur allowed six goals on 21 shots.
Schneider gave up Derek Stepan's penalty-shot goal with 9:54 remaining. The seven goals matched the Rangers' season high.
The Devils grabbed a 1-0 lead 5:36 in when Elias took a pass from Ryane Clowe at the Rangers blue line and came in alone. He snapped a shot that beat Lundqvist inside the left post.
The early edge was short-lived as Moore tied it at 9:07 with his fourth goal. Moore got to a loose puck off a rebound of defenceman Anton Stralman's shot and flipped the puck into the net at the right post.
But Elias struck again just 22 seconds into New Jersey's second power play, with help from Czech Republic Olympic teammate Jagr, and put the Devils back in front with 9:00 minutes left with his ninth goal.
Jagr rushed the puck toward the net at the left post, and got Lundqvist to go down. But instead of shooting, Jagr found Elias driving to the net and fed him a perfect pass for a goal from the top of the crease.
That pushed Jagr past former Pittsburgh Penguins teammate Mario Lemieux for 10th place in the NHL career assist list. Jagr had tied Lemieux on Friday against Washington with another two-assist performance.
Jagr notched assist No. 1,035 when Zajac made it 3-1 with 3:53 left in the first. Mark Fayne fired a drive that struck Jagr in front and bounded toward Zajac, who got the puck in front of the crease and scored his eighth goal.
The Rangers got back within one when Staal turned a faceoff win in the Devils zone into a score when his shot squeezed in between Brodeur's pads with 3:01 remaining. It was Staal's third of the season.
NOTES: Schneider is 5-1-2 with a 0.96 goals-against average and .961 save percentage in his last eight starts, dating to Dec. 28. Brodeur had played in only two of New Jersey's previous eight games. ... New Jersey had won two straight. New York snapped a two-game losing streak.