How Facebook catches would-be child molesters by analyzing relationships and chat content

fingers on keyboard_thumb

Child at computer, courtesy of ShutterstockLaw enforcement is hailing Facebook for using its little-known data monitoring technology to spot a suspicious conversation about sex between a man in his early thirties and a 13-year-old girl from Florida.

According to Reuters, Facebook software on March 9 raised the red flag when it picked up on a conversation about sex between the man and the girl.

The two had only a loose relationship on the network.

The man was chatting about sex with the girl and planned to meet her after middle-school classes the next day, according to Reuters.

The conversation was automatically flagged for Facebook employees, who read it and quickly notified the police.

Police took over the girl’s computer and arrested the man the following day, Special Agent Supervisor Jeffrey Duncan of the Florida Department of Law Enforcement told Reuters.

The alleged predator has pleaded not guilty to charges of soliciting a minor.

Facebook doesn’t talk much about this technology, which scans postings and chats for criminal activity.

Facebook eyeIn what Reuters called the company’s “most expansive comments on the subject to date”, Facebook Chief Security Officer Joe Sullivan said that the monitoring software analyzes relationships to find suspicious conversations between unlikely pairings, i.e., between people of widely varying ages who only have loose and/or newly formed relationships, for example.

The technology also relies on archives of real-life chats that preceded sexual assaults, Sullivan told Reuters.

It’s easy to see why Facebook doesn’t talk about it much: the last thing the company wants is for its users to feel like they’re being eavesdropped on, Sullivan said:

We've never wanted to set up an environment where we have employees looking at private communications, so it's really important that we use technology that has a very low false-positive rate.

To avoid coming off as eavesdroppers, Facebook also avoids probing what it interprets as pre-existing relationships, Sullivan said.

Fingers at keyboard, courtesy of ShutterstockReining in its monitoring technology is understandable in light of not wanting to be perceived as Big Brother, but as Reuters pointed out, a low false-positive rate has the serious downside of letting many dangerous communications go through unflagged.

Duncan estimates that for every predator the police intercept due to tips from Facebook and other companies, another ten get through the system undetected.

And while Facebook limits how visible children are to its adult users – minors don’t show up in public searches, only friends can chat with them, and only friends’ friends can send them messages – children are all too capable of lying about their age and pretending to be adults.

The converse is true: adults can lie about their birth dates and pretend to be minors.

One example can be found in Skout, a location-based social networking mobile app and website that in June barred minors from using its service, following three separate incidents in which children were allegedly sexually assaulted by adults posing as teenagers.

At the time, the New York Times reported that Skout was fully aware that minors were using its site.

Skout had, in fact, put safeguards in place to protect those minors. Last year, after noticing minors using its service, Skout put together a separate service for 13- to 17-year-olds with safety features such as parental controls.

In addition, Skout devoted a quarter of its staff to monitoring activity to flag nudity, and to check chats for inappropriate sexual messages, profanity, spamming, copyright infringement and violent behavior. The service also banned tens of thousands of infringing devices every month.

Skout logoIn spite of Skout’s efforts, three children were allegedly targeted, raped or molested.

There’s no lack of security to protect against the type of age falsification that creates problems on Facebook and sites such as Skout.

Reuters pointed to one such provider, Aristotle International Inc., which offers methods such as having a parent vouch for a child with a token credit card payment.

The problem is, nobody’s buying.

The downsides of such technology: it bleeds away sites’ profits because it costs money, and it drives away children who crave unfettered freedom of communication.

Children’s natural development includes the need to break away from their families as they seek independence.

Tragically, there are no end of online venues that have the look and feel of sanctuaries where it’s safe to do that in the presence of peers.

It’s crucial to somehow get through to them that those sanctuaries can be smoke and mirrors, and that those supposed peers can all too easily be dangerous predators.

Parents, law enforcement, you have my sympathy. The task seems overwhelmingly daunting.

How do you get these lessons through to minors? Please, share your wisdom with us by leaving a comment below.

If you want to learn more about privacy and security threats on the social network and elsewhere on the internet, join the Sophos Facebook page.

Fingers at keyboard and Child at computer images courtesy of Shutterstock