Background
In April, we announced a pilot to address an industry-wide challenge: the lack of non-English grooming detection tools, which leaves minors around the world vulnerable to harm.
The challenge
To address this challenge, we partnered with an APAC-based member company to develop a Korean- and English-language classifier capable of detecting grooming-like behaviors in text, such as discussions of sexual content or statements that indicate the presence of an adult and a minor.
Update
We’re pleased to share that our Korean-language grooming classifier has now been built and is undergoing internal testing. Developing the model was no small feat: we labeled more than 4 million lines and 50,000 conversations with grooming attributes, then collaborated with another member company to translate them from English to Korean using their proprietary in-house translation models.
Building the classifier revealed several key insights, including the challenges of capturing cultural nuances in grooming detection and the large volumes of data required to test such models effectively.
Despite these challenges, early results are promising—the model has performed well in detecting grooming behaviors in both short and long conversations.
What’s next
Next, we’ll continue testing and refining the model with other members, with the goal of sharing this tool more broadly to help protect vulnerable users across languages.
