It was Australia's first loss in the Olympics to a team other than the United States since 1996 and was one of the biggest victories in France's women's basketball history.
Sandrine Gruda added 11 points for France (2-0) in the win of the second-ranked team in the world.
The French didn't make it easy on themselves, failing to put the game away in the final minute of regulation.
Australia capitalized on France's miscues.
France's Isabelle Yacoubou had a chance to seal the victory in regulation, but missed the second of two free throws. Trailing 65-62 with 3 seconds left, Australia's Belinda Snell got the rebound and banked in a 3-pointer from just past halfcourt to send the game to overtime and whip the crowd into a frenzy.
The Australians, however, had to play the extra session without stars Lauren Jackson and Liz Cambage — both had fouled out.
France has been on the rise as a women's basketball power. The French won the European Championship in 2009 and qualified for the Olympics for only the third time. Now they can add a victory over the three-time defending Olympic silver medallists to their growing resume.
The two teams met in an exhibition game just before the Olympic competition began and France came away with a six-point victory. Jackson and Suzy Batkovic didn't play in that contest, but having them back didn't make a difference in the outcome on Monday.
Batkovic led Australia with 17 points and Jackson added 13 before fouling out.
Trailing 61-59 with 1:43 left in the fourth quarter, Celine Dumerc hit a 3-pointer. Gruda extended the lead to three with a lay-in with 47 seconds left.
France just couldn't put Australia away. After Snell missed a 3-pointer that would have tied it, Gomis missed a wide-open layup on the other end.
Snell was fouled on the ensuing rebound. She made the first free throw before missing the second on purpose.
France corralled the rebound, but then Yacoubou's missed free throw set up Snell's thrilling shot.
Australia was hoping this would be its chance to get that elusive gold medal. The team spent the last three months training together as Jackson missed the first part of the WNBA season to work with her teammates. Australia may be hard-pressed to get over the top this time without Penny Taylor, who is missing the Olympics after she tore the anterior cruciate ligament in her left knee in early April.
France turned a 28-27 halftime lead into a 13-point advantage behind Gomis. She took over scoring 16 points during a 21-7 run that gave France a 52-39 lead with 3:37 left in the third quarter.
Australia rallied scoring the final 12 points of the period behind Jenna O'Hea and Jackson to cut its deficit to one heading into the fourth quarter.
France built its lead to five early in the fourth before Australia took a 59-58 lead on Samantha Richards' jumper with 4:11 left.
After Gomis hit a free throw to tie it, Jackson had a basket with 2:48 left.
Jackson helped Australia get out to a quick 18-11 lead before France started to rally behind Edwige Lawson-Wade. She had six straight points during a 10-0 run that gave France its first lead of the game at 21-20 with 4:20 left in the first half.
The teams traded baskets the rest of the half and French held a 28-27 advantage at the break.