Orpik, a defenceman, took a shot from the left point that found its way through traffic and got past goalie Evgeni Nabokov.
Orpik had never scored in 77 previous playoff games and didn't have one in the past 106 contests of any kind since he last scored against the Islanders on Nov. 21, 2011.
He now has 12 career goals in the NHL. Evgeni Malkin moved the puck from behind the net to Tyler Kennedy, who fed it to Orpik for the winning shot.
"I'm definitely not a goal-scorer, but he laid it right on a tee for me," Orpik said. "I wish I can say I was trying to go there, but I was just trying to put it on net, and found a hole."
The Penguins advanced to face the Ottawa Senators in the second round of the playoffs despite being outshot 38-21 in the decisive win.
The Islanders were just 5:16 away from sending the series back to Pittsburgh for one more game when another defenceman, Paul Martin, got the Penguins even for the third time.
"It's great to finish it, I tell you that," goalie Tomas Vokoun said. "We got everything we could have handled. They played great and it was a tough series."
Evgeni Malkin assisted on the tying and winning goals. On Martin's goal, Malkin curled behind the New York net with the puck and sent a hard pass high in the zone to Martin, who ripped a drive through traffic in front.
Michael Grabner had given the Islanders a 3-2 edge 2:21 into the third.
He scored his second career NHL post-season goal off a feed from Keith Aucoin to give the Islanders their third lead — on their 30th shot — against the top-seeded Penguins. The goal left Vokoun sprawled on his stomach.
The teams had alternated wins since the Penguins took the opener 5-0 at home. Pittsburgh got into position to advance to the second round for the first time since 2010 when it won 4-0 in Game 5 with Vokoun in goal instead of Marc-Andre Fleury.
"To be thrown into the middle of a series and play the way he did, that's huge," Penguins captain Sidney Crosby said. "Especially tonight. They carried the play and had a lot of great chances. He was sturdy and solid for us."
Vokoun had trouble early matching the success he had all season against the Islanders, but still pulled out the win. The Penguins had lost all three previous post-season meetings with the Islanders — including two defeats in Game 7.
John Tavares, Colin McDonald and Grabner gave New York leads in each period. Aucoin had two assists, and Nabokov made 17 saves.
Jarome Iginla tied it in the first, and Pascal Dupuis made it 2-2 in the second for Pittsburgh, which got 35 saves from Vokoun.
Despite being outplayed for much of the game — as evident by the shot disparity of 28-13 through two periods — the Penguins entered the third in a 2-2 tie.
Pittsburgh overcame a pair of New York power plays in the second — including one for too many men on the ice that had coach Dan Bylsma irate on the bench — and got even again when Dupuis scored his fifth of the series with 9:01 remaining.
Joe Vitale raced along the right wing boards and flung the puck in front to Dupuis, who redirected it past Nabokov.
Pittsburgh nearly took its first lead of the night in the final minute of the second when Crosby eluded Frans Nielsen behind the New York net and found Chris Kunitz at the right post for a quick one-timer that Nabokov blocked.
The Islanders showed no signs of nervousness or that the moment was too big for them, despite the lack of playoff experience throughout the roster.
New York forced the Penguins into an early icing violation that caused Bylsma to burn his lone timeout just 1:19 in in order to rest the tired players forced to stay on the ice.
The Islanders kept the pressure on, and spent a large chunk of time in the Pittsburgh end — largely in the corners and behind the net — but it paid off with an early goal.
Josh Bailey dug the puck out of the right corner and fed a perfect backhanded pass out to Tavares, who was left alone as he skated into the slot. Tavares grabbed the puck and snapped off a crisp wrist shot from the hashmarks that beat Vokoun at 5:36.
As the Islanders celebrated, Tavares was showered with chants of "M-V-P, M-V-P" one day after he was announced as a finalist for the Hart Trophy.
The goal carried even more significance than the 1-0 lead it created as it was the Islanders' first against Vokoun in two games this series.
Counting the regular season and his shutout win Thursday in Game 5, Vokoun entered with a 4-0 mark, an 0.69 goals-against average, and .977 save percentage this year in five games against the Islanders.
But Crosby, also one of the three Hart finalists, created the tying goal with a strong, unimpeded drive on Nabokov. Crosby surged to the net and put a backhand on the goal. Nabokov made the stop, but couldn't grab the puck before Iginla came into the middle and poked the rebound in at 7:39.
It appeared the game would remain tied until the intermission, however McDonald put the Islanders back in front 2-1 with 37 seconds left in the first.
More hard work on the end boards led to the goal as Grabner fought off Martin and managed to shove the puck out front to Aucoin, who quickly sent a pass across the crease from the left post to the right, where McDonald was left unchecked. In one motion he steered the puck into the open side before Vokoun could recover.
NOTES: Dupuis had five goals and two assists in the series, earning at least one point in all six games. ... C Brock Nelson made his NHL debut for the Islanders. LW Jesse Joensuu, who played in Game 5, sat out. ... Malkin had nine assists in the series.Suggest a correction