A Facial Recognition System Told Delhi Police 25 People Were at a Protest. They Were in Jail.
What happened
Delhi Police deployed a facial recognition system at the Jantar Mantar protests and used it to identify individuals in the crowd. The system flagged at least 25 people. The problem: every one of them was in jail at the time.
Police, prison, and court records all confirmed the same thing. The 25 individuals had verifiable custody records placing them elsewhere. The system did not flag possible matches or low-confidence identifications for human review. It produced outputs that investigators could act on, and those outputs were wrong in a way that any cross-check with existing government databases would have immediately surfaced. That cross-check did not happen before the identifications were recorded.
For the people flagged, the implications are serious. A protest identification can become the basis for questioning, arrest, or criminal proceedings. In a legal system that moves slowly, a false identification embedded in a police record at the outset can follow an individual long after the underlying error is discovered. The 25 people involved had verifiable proof of their whereabouts. Many people in similar circumstances would not.
The deeper problem is not that the system produced false matches. All facial recognition systems produce false matches, and the error rate for this type of identification is well-documented, rising sharply across certain populations and image conditions. The problem is that output from the system reached active investigation without a mandatory verification step: no check against incarceration records, no second reviewer, no documented confidence threshold before an identification was treated as actionable. The system was used as if its output were a finding rather than a candidate requiring confirmation.
There is no indication in the record that the system logged which individuals it flagged, what confidence scores accompanied those flags, or whether any human reviewer approved the identifications before they were recorded as part of protest documentation. That is the accountability gap this kind of deployment creates: a provable record of what a system did, what it decided, and who verified it before any person's legal status was affected does not appear to exist. Without that record, the 25 false identifications are an embarrassment. With it, they would be evidence of exactly where the process broke down.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.