After Ibrahimovic had given Sweden the lead in the 52nd minute, Shevchenko responded by heading in the equalizer just three minutes later and then added the winner in the 62nd.
"We showed today our character, our good football," Shevchenko said. "We have very good chances to qualify."
The victory gives Ukraine a serious chance of advancing from Group D after England and France drew 1-1, while Sweden faces an uphill battle to remain in the tournament.
Shevchenko is already a national icon in Ukraine and carried the hopes of the team going into the tournament. In front of sea of yellow and blue in the packed Olympic Stadium, he certainly didn't disappoint.
Showing flashes of the scorer's instinct that once made him one of the world's top strikers, Shevchenko beat Olof Mellberg to a cross for his first goal and then met a corner to head the ball inside the near post for the second, setting of shouts of "Sheva" that will likely reverberate throughout the nation for a long time.
These days it's Ibrahimovic who is considered a world-class forward, but AC Milan's current star came up short against its former.
Ibrahimovic did provide the focal point of the Swedish attack and shaved the post with a header in the first half before putting his team ahead.
Kim Kallstrom collected a long cross at the left side of the area and picked up Ibrahimovic in front of the net, and the Sweden striker calmly slotted the ball past Andriy Pyatov and into the net.
"I feel sad. We had more chances to score," Ibrahimovic said. "It wasn't supposed to happen. The situation is as it is. There's still two games left and we still have the two games to get a result. We need to play a better game than we did today.
"We can't really lose the lead like we did."
Sweden seemed to relax after the goal and Shevchenko took advantage by meeting Oleh Gusev's cross and sending his header past Andreas Isaksson.
That made the stadium explode, and it grew even louder six minutes later as the 35-year-old Shevchenko's header from a corner found its way past defender Mikael Lustig at the near post, sending coach Oleg Blokhin jumping up and down the sideline and waving his hand frenetically.
Ibrahimovic had one good chance to equalize but his hard swerving shot was saved by Pyatov, while substitute Johan Elmander and Mellberg both shot high in the final minutes.
It was clear early on that both teams were eager to press for a win, knowing that tougher opposition was ahead in the group.
Sweden coach Erik Hamren opted for an unabashedly attack-minded lineup with Markus Rosenberg playing up front with Ibrahimovic behind him and with fellow striker Ola Toivonen on the left wing.
However, the Swedes failed to impose their will on the game in the first half as Ukraine came out looking the more ambitious.
Shevchenko showed his intentions by starting a quick counterattack in the 23rd and got the ball back as he broke into the right side of the area, but his shot from a tight angle rolled wide.
At the other end, Ibrahimovic came closest for Sweden when he was picked out unmarked in the area by Sebastian Larsson, but his header in the 39th went off the outside of the post.
"Today it was off the post and out, the next game it will be off the post and in," Ibrahimovic said. "We have three games to play here, it's not just one game."
After Ukraine took the lead, Hamren brought on veteran Anders Svensson, Christian Wilhelmsson and Elmander but Sweden couldn't force an equalizer.
Blokhin, meanwhile, decided against resting Shevchenko ahead of the games against France and England, even though the forward has said he doesn't think he can play three games in nine days.
Turns out that was a pretty good decision.
Shevchenko was given a well-deserved standing ovation as was taken off in the 81st, while Ibrahimovic strode off the field immediately after the final whistle with his head down.
"Every game for us is like a final," Shevchenko said. "We showed great football tonight."
Ukraine: Andriy Pyatov, Oleh Gusev, Taras Mykhalyk, Yevhen Khacheridi, Yevhen Selin, Andriy Yarmolenko, Anatoliy Tymoschuk, Yevhen Konoplianka, Serhiy Nazarenko, Andriy Voronin (Ruslan Rotan, 85), Andriy Shevchenko (Artem Milevskiy, 81).
Sweden: Andreas Isaksson, Mikael Lustig, Olof Mellberg, Andreas Granqvist, Martin Olsson, Rasmus Elm, Kim Kallstrom, Sebastian Larsson (Christian Wilhelmsson, 68), Zlatan Ibrahimovic, Ola Toivonen (Anders Svensson, 62), Markus Rosenberg (Johan Elmander, 71).Suggest a correction