Swiss rider Fabian Cancellara avoided the trouble and retained the overall lead for a fifth day after the 214.5-kilometre trek alongside the English Channel from Abbeville to Rouen.
The top standings didn't change: the Swiss leads second-placed Bradley Wiggins, who hopes to be Britain's first Tour winner, by seven seconds. Defending champion Cadel Evans of Australia was 17 seconds off the pace in 7th place.
With less than three kilometres left, Cavendish went down in a crash - scraping up his rainbow-coloured jersey of world champion. He got back on his bicycle and rode gingerly to finish the stage. His Team Sky said he was scraped up, but appeared to have no serious injuries.
With Cavendish out of the picture, Greipel burst out of the depleted group of sprinters, and sped to the straightaway finish, a split second ahead of Italy's Alessandro Petacchi and Dutch rider Tom Veelers.
"This is what we wanted. It's a good victory," said Greipel after his 14th victory in all competitions this year. He said he didn't pay much attention to the late crash.
"I heard something behind me ... but 60 kilometres per hour, you don't worry about what happened behind," the Lotto-Belisol rider said in an interview with France-2 TV.
Despite the crash, Greipel said his victory was well-deserved.
"There were still really fast guys there for the sprint and I think we just deserve this victory," he said, playing down a question about whether he savoured it less because Cavendish was knocked out.
"I think it's no question about that," Greipel said. "I won a stage in the Tour de France!"
Dave Brailsford, manager of Team Sky, said Cavendish's injuries were "more superficial, just skin. We'll see later. After a crash like that, it's high emotion."
"When you hit the ground that fast, it shakes you up, it builds up adrenalin," he said. "We'll let the adrenalin ... calm down and see where we are later. Now he is OK. He is having a shower. It should be OK tomorrow."
The group spill also brought down riders like Garmin-Sharp's Robbie Hunter, and Cavendish's lead-out man on Sky, Bernhard Eisel of Austria — and Brailsford hinted at his rider's mood afterward.
"I can't repeat what he said when they came into the bus," he told French television.
Victoria's Ryder Hesjedal finished 45th in the stage and fell one place in the overall classification to ninth.
Cavendish, who was voted the BBC's sports personality of the year in Britain last year, won Monday's second stage in a sprint — giving him a career 21st Tour stage victory.
An official race medical report said Cavendish sustained several scratches and a cut on a finger, Eisel sustained a gash that required stitches on his forehead, and Hunter scraped up his left side. Daniel Oss of Liquigas had a hip injury, it said.
According to the Tour rulebook, riders who get delayed by a crash in the last three kilometres of the stage are awarded the same time as the stage winner.
The pack clocked the same time as Greipel — 5 hours, 18 minutes, 32 seconds — though some stragglers nursing wounds from crashes earlier this week, like world time-trial champion Tony Martin of Germany, and Tom Danielson of the United States, straggled in 2:21 behind.
Cancellara, who briefly got stalled by the crash, sighed with relief: "I'm really happy to get past that, a fall early hurts ... today it was calm, and then hectic at the finish."
David Moncoutie and Anthony Delaplace of France and Japan's Yukiya Arashiro broke away early and chiselled out a maximum lead of 8:40 at kilometre 16, but the pack reeled them in with about 10km left in the stage.
Riders set off from Abbeville - a town where 6,000 buildings were destroyed or damaged in a German bombing raid in World War II in May 1940 - and rode along the English Channel's picturesque, chalky cliffs with views of giant wind-turbine installations.
Thursday's fifth stage promises another bunch sprint, after a mostly flat 196.5-kilometre course from Rouen to Saint-Quentin north of Paris.