IOC medical commission chairman Arne Ljungqvist was sharply criticized by senior IOC member Dick Pound on Wednesday for only deciding in May to retest about 100 samples from Athens.
The International Olympic Committee is investigating up to five possible positive results from those retests.
Pound, former chairman of the World Anti-Doping Agency, expressed "concern and disappointment" about the IOC's reluctance to retest earlier — an unusual public rebuke during the IOC's general assembly.
"The keeping of samples was meant to be a major deterrent," Pound said. "If those we hope to deter understand we do nothing, there is not going to be much of a deterrent."
"If we're going to live up to our objective of zero tolerance in doping, we have to take advantage that we have all these samples and take the opportunity to retest them," he added.
Pound also complained that the IOC had only decided to carry out the retests after pressure from the media and WADA.
Ljungqvist agreed the IOC could have moved sooner but said the committee had no evidence that any banned substances used in Athens hadn't already been tested for at the time.
"I admit we could have done it a little earlier," he said.
The IOC stores doping samples from each Olympics for eight years to allow for retesting. The statute of limitations for Athens will expire Aug. 29, the date the games closed in 2004.
In a separate interview with The Associated Press, Ljungqvist defended the timing on grounds that testing methods are more effective now.
"The longer you wait the better if you want to catch someone," he said.
In the session, Ljungqvist noted that the IOC did test samples retroactively from the 2008 Beijing Olympics and 2006 Winter Games in Turin to check for the use of blood-boosting drug CERA. Those retests led to five new positive cases from Beijing — including the stripping of Rashid Ramzi's gold medal in the 1,500 metres — but none from Turin.
"With respect to the Athens Games, there was no information that any substance would have been in use that was not already analyzed for," Ljungqvist told Pound. "We waited quite a while. The idea came up I admit quite late."
He said today's testing methods are "more sophisticated and sensitive" than they were eight years ago.
"Had we done it earlier, maybe halfway through, we probably would not have found much," Ljungqvist said. "Now we have done some retests and we do have a few reports from the lab that are suspicious."
The IOC has declined to identify which athletes, sports or substances were flagged up, pending testing of the backup "B'' samples.
If positive cases are confirmed, the IOC could retroactively disqualify athletes, nullify results and strip medals.
In 2004, the Athens Games produced a record 26 doping cases, more than double the previous Olympic high of 12 in Los Angeles in 1984. Six medallists , including two gold winners, were caught.
Ljungqvist took up Pound on his suggestion that the IOC medical commission formulate a policy on future retesting.
"This is a good lesson," Ljungqvist said. "We'll follow your advice and together with WADA develop a strategy for further use of stored samples."
On a separate issue, Ljungqvist said the IOC has set up a three-person medical panel in London to investigate any cases of female athletes suspected of having excessive levels of male hormones.
For the first time, the IOC is enforcing rules determining the eligibility of women athletes with "hyperandrogenism" — a condition involving overproduction of male sex hormones. The rules were approved last year.
Ljungqvist said the panel consists of an endocrinologist, geneticist and gynecologist.
"We have this panel standing by should a case come up that needs to be investigated," he said. "The national Olympic committees have been informed ahead of time. They have the duty to make sure the entries are competing in the right category."
At the heart of the matter is whether a female athlete derives a competitive advantage over other women because of higher than normal levels of hormones such as testosterone.
The issue of gender verification gained global attention after South African runner Caster Semenya was ordered to undergo sex tests after winning the 800 metres at the 2009 world championships. The IOC said the Olympic rules were not related to the Semenya case.