Committee chief Ahmed Nabil al-Taher al-Alam was released unharmed a week after unknown gunmen abducted him from his car in the capital.
"It will be become clearer in the coming days. They raised no issue and they made no requests," he told The Associated Press.
Al-Alam said his kidnappers posed as officials when they took him near his office on Tripoli. They later released him in Misrata, and rebels helped bring him back to Tripoli in co-ordination with security officials.
"It is a mystery. Investigation is promised and may this be a good omen to end these kidnappings," he said.
The abduction comes during a wave of score-settling among rival groups left over from Libya's eight-month civil war that ended with the capture and killing of longtime dictator Moammar Gadhafi in October.
Dozens of former rebel groups now operate independently, often ignoring the nation's weak central government.
Since Gadhafi's fall, militias frequently have taken matters into their own hands and rounded up officials with ties to the ousted regime.
Salah al-Alam, brother of the official who was kidnapped, said it was not clear why his brother was abducted or on what conditions his abductors let him go. He said they are suspected to be a criminal gang, and had interrogated his brother for only 15 minutes during his week of captivity.
Security officials were not immediately available for comment.
Al-Alam served as head of the Libyan Football Association under Gadhafi. He is known to have been friendly with Gadhafi's son, Mohammed, who was al-Alam's predecessor as Olympic Committee president.
Five Libyans have qualified to compete in the 2012 Summer Olympics to be held in London. A committee official said the delegation left for London Saturday.
"We're tremendously relieved," International Olympic Committee spokesman Mark Adams said. "We were very concerned. We're very happy it's come to a successful conclusion."
Al-Alam said he will be joining the Libyan delegation in London soon.
AP Sports writer Stephen Wilson in London contributed to this report.