Speaking out publicly by name for the first time, Aaron Fisher said in an interview airing Friday on ABC's "20/20" that the Pennsylvania attorney general's office had told him it needed more victims before Sandusky would be charged.
Fisher first reported the abuse in 2008. Sandusky was arrested last November. Fisher said the delay made him increasingly desperate.
"I thought maybe it would be easier to take myself out of the equation," he told ABC. "Let somebody else deal with it."
Fisher, now 18, testified as Victim 1 at Sandusky's trial. He, his mother and his psychologist have co-written a forthcoming book about his ordeal.
Fisher told jurors that Sandusky approached him through a summer camp for youth sponsored by The Second Mile, a charity for at-risk youth the former coach had founded.
Physical contact began with a hand on his leg in the car, Fisher said, and he began spending nights at the Sandusky home in State College, about 30 miles from his own home in Lock Haven. Kissing and back rubbing during those overnight visits progressed to oral sex. He said he tried to distance himself from Sandusky, to no avail.
Fisher was 15 when and his mother eventually reported the abuse to the school principal, who responded that "Jerry has a heart of gold and that he wouldn't do those type of things," Fisher told ABC, repeating his trial testimony.
"They tell me to go home and think about it," he told ABC.
School officials reported Sandusky to Clinton County Children and Youth Services, which began an investigation.
As Fisher spoke out, Sandusky launched his effort to overturn his conviction, contending there wasn't enough evidence against him and the trial wasn't fair. His lawyers filed a 31-page document Thursday that attacked rulings by the judge, the closing argument by the prosecution and the speed by which he went from arrest to trial.
Sandusky wants the charges tossed out "and/or" a new trial, saying the statute of limitations had run out for many of the 45 counts for which he was convicted in June. Currently in a county jail near State College, he is awaiting transfer to the state prison system to begin serving a 30- to 60-year sentence.
"The defendant submits the court's sentence was excessive and tantamount ... to a life sentence, which the defendant submits is in violation of his rights," his lawyers wrote.
A spokesman for the attorney general's office said the Sandusky filing was under review.
Fisher and seven other young men testified against him in June, describing a range of abuse they said included fondling and oral and anal sex when they were boys.
Sandusky didn't testify at his trial but has maintained his innocence in interviews and at sentencing.
Sandusky, 68, built a reputation as one of the country's premier defensive coaches while serving under head coach Joe Paterno, including two national championships. That image was shattered last year by his arrest.
The abuse scandal rocked Penn State, bringing down Paterno and the university's president and leading the NCAA, college sports' governing body, to levy unprecedented sanctions against the university's football program.
Two Penn State administrators were charged as a result of the investigation into the Sandusky allegations, accused of lying to the grand jury that investigated Sandusky and not reporting suspected child abuse to the proper authorities. Those two officials, athletic director Tim Curley, who is on administrative leave, and retired vice-president Gary Schultz, await trial in January and maintain their innocence.
Former FBI Director Louis Freeh, hired by university trustees to conduct an investigation into the university's handling of abuse complaints against Sandusky, concluded that Paterno, who died in January, along with ousted President Graham Spanier, Curley and Schultz concealed a 2001 allegation against Sandusky to protect Penn State from bad publicity.
The late coach's family, as well as Spanier, Curley and Schultz, dispute Freeh's assertions.