Bayern defender Jerome Boateng's powerful shot appeared to deflect off teammate Mario Goetze, rising up and beyond the dive of his former City teammate Joe Hart, who had denied the 2013 champion with a series of saves.
The goal brought Bayern coach Pep Guardiola racing down the touchline to celebrate with Boateng.
"We know how difficult the group is and a victory for the start is very, very important for us," Guardiola said.
Barcelona and new coach Luis Enrique had a more routine night, with the Catalan side scoring in the 28th minute when defender Gerard Pique headed in Lionel Messi's free kick.
"We have to improve a lot, but I am very happy," Luis Enrique said.
Chelsea was held 1-1 at home by Schalke after Cesc Fabregas's 11th-minute opener for the hosts.
On a bad night for eastern European champions, Roma routed CSKA Moscow 5-1 and Porto crushed BATE Borisov 6-0.
CSKA faces further sanctions from UEFA after its fans threw flares, holding up play for two minutes in the second half at the Olympic Stadium. It must already host Bayern next in an empty stadium because of fans' racist behaviour last season.
Elsewhere, Ajax drew 1-1 with Paris Saint-Germain, Maribor grabbed an equalizer in injury time to draw 1-1 with Sporting Lisbon and Athletic Bilbao's game against Shakhtar Donetsk finished 0-0.
Bayern's late winner reprieved Thomas Mueller for missing the German champion's best chance in the first minute of a lively match.
Mueller wasted an open-goal chance, rounding Hart before losing balance as he shot wide of the near post.
"As long as the team wins, I'll lie in bed tonight with a smile," Mueller said.
Hart excelled in the first half pushing away Mueller's powerful header and shots from Mario Goetze and David Alaba.
The England goalie saved his best effort to deny Boateng's long-range shot one minute before the goal came.
In a strong Group E, Roma tops the standings after Gervinho scored twice and another former Man City defender, Brazil's Maicon, were on target against the Russian champion.
Barcelona heads Group F after PSG failed to exploit its superiority on Zlatan Ibrahimovic's return to Ajax, where he made his reputation in a three-season spell a decade ago.
The French champion led in the 14th when Uruguay forward Edinson Cavani pounced after Ibrahimovic's shot was blocked on the line.
Ajax levelled with its first shot on target in the 74th as Denmark midfielder Lasse Schoene curled in a free kick which goalie Salvatore Sirigu pushed into the net. Sirigu did better with Schoene's next effort which he turned on to the post.
Chelsea has a 100 per cent record in the Premier League and seemed to be cruising when Fabregas finished off a reverse return pass from Eden Hazard.
However, the 2012 winner paid for misses by Didier Drogba, whose penalty in the shootout won that final against Bayern in Munich, replacing the in-form Diego Costa.
Schalke levelled in the 62nd when Fabregas lost the ball and a counterattack ended with Netherlands forward Klaas-Jan Huntelaar's goal.
"A good point for them. Not a fantastic point for us, but a point," Chelsea coach Jose Mourinho said. "We had so many great situations. Again, that's football. To defend like they did is also football."
That left Group G all tied up after Maribor levelled in stoppage time against Sporting, through substitute Luka Zahovic.
The Portuguese visitors seemed to have won when former Manchester United winger Nani fired in a left-foot shot in the 80th.
Porto winger Yacine Brahimi starred on his Champions League group-stage debut after impressing for Algeria at the World Cup.
Brahimi scored a first career hat trick before being substituted to a standing ovation in a six-goal romp against the champion of Belarus. Jackson Martinez, the Colombia forward, Adrian Lopez and Vincent Aboubakar also scored.
Porto's creativity made up for Group H rivals Bilbao and Shakhtar failing to shine in their scoreless draw.