Ottawa police said they started an investigation in November after a complaint of sexual assault against a minor by a teacher "during the exercise of his functions as a teacher for Grade 7 and 8."
They say there are concerns there may be other victims.
Phillip Nolan, an elementary school music teacher, has been charged in connection with incidents dating back to 1998 and 2000, the police said.
During the investigation, Nolan was removed from the classroom at Avalon Public School in the city's east end and placed on administrative duties, the Ottawa-Carleton District School Board said in a statement. He has now been suspended.
"This is an extremely difficult situation and as a parent myself, I share the concerns that our community is feeling," said Jennifer Adams, the board's director of education.
"The district has resources available to support parents and students at the school level and on our district website."
The board declined further comment.
Harper's office appeared genuinely stunned by the development.
"We are obviously shocked and disturbed by these serious charges," communications director Jason MacDonald said in a statement.
"No allegations of this nature have ever been brought to our attention. Anyone found guilty of these charges should face the full force of the law. Given that this individual is now facing charges, it would be inappropriate to comment any further at this time."
Conservative MPs reacted to news of the arrest with disbelief Thursday as they emerged from the House of Commons.
"I am shocked, but it's up to the police to deal with this matter," said Natural Resources Minister Joe Oliver, who said he never met Nolan.
"I think he should be treated with the full force of the law like anybody else, if found guilty," said Alberta's Leon Benoit. "Let him have his due process and let him face the law."
But Liberal MP John McCallum said the arrest has nothing to do with the prime minister, and the police investigation should not have any bearing on Harper.
"I think it’s independent of the prime minister," McCallum said outside the Commons.
"I think people we come in contact with sometimes get into trouble and it's not necessarily anything to do with us. So I would treat it like another case that is before the courts."
Harper has known Nolan and the other band members for years; they have played at the prime minister's residence at 24 Sussex Drive. They also played together at the National Arts Centre in 2009, when Harper made his unexpected musical debut.
More recently, Harper and Herringbone played at a Jewish National Fund dinner in Toronto on Dec. 1 of last year, apparently after the police began their investigation.
During that performance, the prime minister described Nolan as a "mild-mannered schoolteacher" and spoke, tongue firmly in cheek, of meeting the band.
"I was a young MP on a ride-along with the city of Ottawa police and they were wondering about these teachers and IT professionals who were respectable, middle-class guys by day, and playing in these seedy bars by night," Harper said, tongue-in-cheek.
"But it turns out the police had nothing on Herringbone, so I joined."
The band's website says Nolan has been involved in the music industry since getting a bachelor of music degree from Queens University in 1994. He has worked in musical theatre, both as a musician and as an actor, dancer and singer. He has also directed award-winning school bands.
Nolan "is also the 2005 recipient of the Arts Educator of the Year award given by the Ottawa-Carleton District School Board arts advisory," the website says. "He won that award previously in 1996 with the Ontario Board of Education."
Also on HuffPost