Phelps has yet to have the quickest time in the morning heats at the London Games four years after he won a record eight gold medals in Beijing.
He finished third in his heat at one minute 55.53 seconds, trailing Dinko Jukic of Austria and U.S. teammate Tyler Clary to the wall.
"I'm pretty happy with that swim," Phelps said. "That's all I needed it to be."
Phelps has won the 200 fly at the last two Olympics and set the world record at the 2009 world championships.
He was back in the water hours after swimming a solid second leg on the 4x100 freestyle relay that took silver after Ryan Lochte was overhauled by the French on the closing lap.
"Just getting last night sort of out of the way was something that I needed," Phelps said. "I guess I got to bed pretty late. We're all falling asleep pretty late. We're all excited."
Phelps' mother, Debbie, nodded her head slightly when she saw her son had safely advanced. He was the slowest qualifier for the 400 individual medley final on Saturday, and his surprising fourth-place finish kept him off the medals podium.
"I have no idea what it was, but I don't think it had anything to do with confidence," he said. "I felt I was ready to swim faster than that. It just didn't happen."
Jukic had the top time of 1:54.79, while Clary was second-best at 1:54.96 in advancing to the evening semifinals.
"I felt fantastic. The time was faster than it was at trials," Clary said. "I hurt a heck of a lot more at the end of the race in trials. So I had actually said to myself coming into the third wall that I was amazed at how well I felt. Just wanted to see what I could bring it home in."
Velimir Stjepanovic of Serbia was third at 1:54.99. Others who made the semis included Chad le Clos of South Africa (fourth), 2008 bronze medallist Takeshi Matsuda of Japan (eighth), 2008 silver medallist Laszlo Cseh of Hungary (ninth), Wu Peng of China (10th) and Nick D'Arcy of Australia (12th).
David Sharpe of Halifax finished seventh in his heat.
Federica Pellegrini qualified fastest in the 200 freestyle, keeping the Italian on course to defend her Olympic title.
Pellegrini touched in 1:57.16 in the third of five heats, with U.S. teenager Missy Franklin swimming in the lane next to her. But the other American, Allison Schmitt, had the second-quickest time of 1:57.33 in advancing to the evening semifinals on the third day of swimming competition.
"It definitely feels like I'm ready to swim," Schmitt said. "I've been preparing for this meet all year. It's been fun so far and I'm looking forward to having more fun tonight."
Franklin was third best in 1:57.62. She led the first 150 metres before Pellegrini surged ahead over the final lap.
"I didn't expect to be that fast this morning," Franklin said. "It's a stacked event. It's definitely going to be fun. I need to keep my energy up. It's going to be tough for sure."
Pellegrini is the world champion and world-record holder in the event. She finished fourth in the 400 free Sunday.
"I'm happy with the time," Pellegrini said. "The bitterness from last night ended right after the race. I know I gave my all."
Montrealers Barbara Jardin and Samantha Cheverton both advanced to the semis after finishing fourth and fifth, respectively, in their heat.
Also moving on were Sarah Sjostrom of Sweden (tied for seventh), Australians Bronte Barratt (10th) and Kylie Palmer (11th), and 400 free gold medallist Camille Muffat of France (12th).
Muffat was tired after a fitful night of sleep.
"I kept getting up and opening the drawer the medal was in to check it was real and not a dream. It is so beautiful," she said. "This event and the relay is just a bonus for me now."
Ye Shiwen of China dominated the 200 individual medley heats. Her time of 2:08.90 was 1.61 seconds faster than 2008 silver medallist Kirsty Coventry of Zimbabwe, who swam 2:10.51.
Ye, the 16-year-old world champion, will be looking to add the 200 title to the 400 IM gold medal she won on Saturday in world-record time.
"She comes home like a gun in that freestyle," said defending Olympic champion Stephanie Rice, who was ninth.
"If I want to have any chance I need to be well up in front of her heading into the freestyle leg, which shouldn't be too hard for me. She's insane at freestyle but I think my other strokes are better than hers."
Caitlin Leverenz of the U.S. was third at 2:10.63, followed by Hungary's Katinka Hosszu in 2:10.68. American Ariana Kukors was seventh in making the 16-woman semifinal.
Calgary's Erica Morningstar did not advance after finishing last.