Police say the refs were accosted throughout the teenage women's game last Sunday, with the male spectator then confronting them after they got off the ice.
It's alleged he threatened to attack one of them, hurled insults at the pair and then tried to enter their changing room.
Police say the referees believed they were about to be assaulted and locked themselves inside to keep the spectator out.
Investigators say he continued to yell and swear at them in full view of others attending the game, including children.
A 47-year-old man was arrested Saturday and is charged with uttering threats of bodily harm, assault and causing a disturbance. His name has not been released.