Matt Anderson added 15 points in the 25-23, 25-16, 25-20 win for the defending Olympic champions. The Germans were led by Georg Grozer with 12.
The U.S. held Germany at bay for most over the first half, but the Germans went ahead 22-21 when Donald Suxho's return caromed out. The Americans quickly regained the lead and scored the final three points, capped by Matt Anderson's kill.
Germany kept up in the second set until Reid Priddy's ace put the U.S. up 14-10. Anderson stretched the lead to 20-14 with another ace, clocked at more than 117 kph, and Priddy's kill finished it.
The U.S. rode the momentum through the third set, taking the match on Grozer's serve into the net.
"Luckily all of us are playing really well right now," Anderson said. "We're playing pretty sound volleyball."
The U.S, ranked No. 5 by volleyball's governing body, is in a difficult pool in London, joining top-ranked Brazil, perennial powerhouse Russia, Serbia, Germany and Tunisia. In the other pool are Italy, Poland, Argentina, Bulgaria, Australia and host Britain.
The top four teams from each group in the preliminary round will advance to the quarterfinals, and the gold medal match in London is set for Aug. 12.
Brazil has struggled of late, and the team failed to make the semifinals in the recent FIVB World League for the first time since 1998. The Brazilians, who swept Tunisia in the Olympic tournament's opening match, face No. 2 Russia in a late match Tuesday at Earls Court.
Germany, ranked 13th in the world, fell to 0-2 with an opening loss to Russia. Up next is Serbia on Thursday.
"It's absolutely a final for us and we must win this match to have any chance of going forward," said German wing spiker Sebastian Schwarz.
The last time Germany and the U.S. played, the Germans came out on top.
Earlier in the day, Bulgaria surprised Poland 3-1. Tsvetan Sokolov had 28 points, including four aces, in ninth-ranked Bulgaria's 25-22, 29-27, 13-25, 25-23 win to go to 2-0 in pool play.
It was the first real upset on the men's side.
Zbigniew Bartman had 21 points for No. 3 Poland, which dropped to 1-1 in the pool after defeating Italy 3-1 in the opener.
Earls Court was filled with red-clad Team Poland fans and festooned with hundreds of Polish flags, many inscribed with the names of different cities.
The fans were quieted when Bulgaria took the first two sets. But Poland rebounded in a big way to dominate the third and deny Bulgaria the sweep.
Bulgaria held off Poland in the final set, winning the mach on Viktor Yosifov's kill.
"Today you have seen a different Bulgaria" Sokolov said.
The Polish players appeared frustrated throughout the match. Poland has been on the upswing since hiring accomplished Italian coach Andrea Anastasi early last year. The team won the recent World League, beating top-ranked Brazil three times in the early round and again in the finals.
Poland, which won the Olympic gold medal in 1976, finished fifth in Beijing.
The Bulgarian team came to London in disarray. After the team earned an Olympic berth at a qualification tournament in Sofia, the coach and a top player suddenly resigned to protest their country's leadership of the sport.
Coach Radostin Stoychev and spiker Matey Kaziyski have called for the resignation of Bulgarian federation chief Dancho Lazarov.
"I am really proud that in this moment of chaos, with coaches coming and going, we're showing our character. This is something which fits very well to the nature and spirit of Bulgarian people," team captain Vladimir Nikolov said.
Bulgaria was also involved in controversy in Beijing, when team captain Plamen Konstantinov suddenly went home for a drug test, spurring rumours about possible doping. He missed three matches.
In other matches Tuesday, Serbia defeated Tunisia 3-1 (25-15, 25-21, 20-25, 25-18), and Italy downed Argentina 3-1 (25-17-21-25 25-17 25-21).