Submit incident
Documented

A Face-Matching App Meant for Fun Became a Tool for Mass Harassment

April 9, 2016
Curated by Team Raidu · Reviewed by Shiva Ganesh
aiid:315View source ↗
LinkedInX

What happened

On the surface, FindFace looked like a party trick. Snap a photo of a stranger on the street, run it through the app, and it would surface their profile on Vkontakte, Russia's dominant social network. That trick turned dangerous in April 2016. Users began pointing FindFace at women who appeared in pornography, tracing their faces back to real names, families, and workplaces, then using that information to shame and threaten them.

The technical achievement here was real. FindFace could cross-reference a face against a social network with hundreds of millions of accounts and return a match in seconds. But nothing about the product asked who was doing the searching or why. A tool built to reconnect people at parties worked just as well for stalkers hunting sex workers and adult film performers, and the company had no mechanism to tell the difference between the two.

That gap is the actual story. Facial recognition systems built for identification are neutral only in the sense that a lockpick is neutral. The moment a service can turn an anonymous photo into a name and an address, it hands that power to whoever opens the app, without regard for consent, intent, or harm. Women who had no reason to expect exposure suddenly found their offline identities attached to content they never meant to make public. Some faced harassment campaigns that followed them into their daily lives.

What's missing from this incident isn't a better algorithm. It's a record of who searched whom, when, and for what stated purpose. Without that, a company can point to broad usage numbers and call the product a success, while the people harmed by it have no way to prove what happened or hold anyone accountable for it.

Reported impact

Affected parties
Not publicly disclosed
Harm type
Not publicly disclosed
Scale
Not publicly disclosed
Financial impact
Not publicly disclosed
Regulatory action
Not publicly disclosed

Classification

Organization
Not publicly disclosed
AI system
Not publicly disclosed
Industry
Not publicly disclosed
Country
Not publicly disclosed
Provider
Not publicly disclosed
Incident type
Not publicly disclosed

Relevant governance controls

Governance control mapping is not available for this record.

  • No controls mappedNot publicly disclosed

Control mapping is analytical. It does not state that any control would have prevented the incident.

Sources and evidence

This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.

AI Incident Database
Also catalogued in
A Face-Matching App Meant for Fun Became a Tool for Mass Harassment
2016-04-09