The firm's technology advances a global campaign against digital abuse in part spurred by the 2012 suicide of Amanda Todd, a teenager from Port Coquitlam, B.C., who was victimized by online sexual exploitation.
"We want to get rid of bullying across the entire Internet."
Millions of users protectedAn internal database query by the firm estimated it has protected at least 34 million users over a recent two-week period in its U.S. data centre alone. Online cruelty inflicted on a Kelowna teenager was also part of the impetus for Community Sift, Priebe said. The teenager was goaded into uploading a selfie that trolls turned against her, generating pages of comments urging her to kill herself. The technology sifts the posts to emphasize positive comments from the 40 per cent of online users who are normally well-behaved to derail the attacks. "They're going to say, 'You're beautiful, you're wonderful, you're helpful,'" Priebe said. "Now she'll have two voices inside her head and she can build the ability to handle all this bullying." The firm builds reputations for users participating online, and detects when someone crosses into a high-risk threshold. Consequences may include limiting identified trolls to certain queues where a moderator can decide if the content is inflammatory, silencing them automatically or banning them outright.
Others have also taken up the cause. A 13-year-old Illinois girl designed software that detects hurtful language as a Google Global Science Fair project. Trisha Prabhu's program ReThink prompts posters to think twice before hitting send. She found more than 93 per cent of teens alter their posts. Programmers with the National Youth Mental Health Foundation in Australia have also developed a Google extension called "reword" that flags potential insults by crossing them out with a red line. Community Sift identifies the tone of online communities rather than policing the Internet, Priebe said. It gives users options to choose settings for avoiding unwanted content, in the same way moviegoers can select films based on ratings. An emerging social world, called Medium.com, has deployed Community Sift to protect its users as they interact and post personal stories. "We want to provide the best place for people to freely and openly express themselves," said Greg Gueldner, who implements the startup's trust and safety protocol.
"We always joke you can put them in the basement with all the other trolls and let them harass themselves."
Safe virtual world for kidsPriebe has boosted online safety before by co-founding Club Penguin, a virtual world where it's safe for children to play games and interact. The company partnered with Disney in 2007 and has a user base of 300 million. The B.C. programmer, who has his own painful story about being bullied into his teens, said people currently believe they're powerless against trolls. "When people realize that it's a solvable problem," he said, "they won't put up with it anymore."
Also on HuffPost