Researchers Studied 26 AI Ethics Staffers. The Pattern That Emerged Should Worry Every Tech Company
What happened
Tech companies love announcing fairness commitments. Far fewer give the people hired to enforce those commitments any actual power to stop a bad system from shipping.
That gap is the subject of research published by Bogdana Rakova, working with Partnership on AI, Spotify, and Accenture. The team interviewed 26 practitioners who worked directly on responsible AI projects inside their organizations, conducting the interviews in late 2019 and publishing the findings in December 2020. By the time the paper came out, several of those interviewees had already left the companies they described. Some told researchers the work itself had taken a toll on them. The timing is hard to ignore: the paper landed the same season Google fired Margaret Mitchell, co-lead of its Ethical AI team, following the ouster of her colleague Timnit Gebru.
The researchers identified a specific structural failure. Employees tasked with catching algorithmic harm typically have no authority to actually halt a project, no reliable channel to escalate concerns, and no clear process for weighing fairness against other business priorities. A follow-up workshop, built around the Two-Loops Theory of Change, asked participants to name what would fix this. Four answers kept surfacing. Give practitioners real veto power over systems, one that cannot be quietly overruled by a product team. Balance internal reviewers, who have more information, against external watchdogs, who can apply pressure without fear of losing their job. Build actual communication channels, like recurring town halls, between employees and leadership. And recognize that none of this works in isolation. Whistleblower protections mean little without a culture that makes people feel safe using them.
That last point is the real finding here. Responsible AI failures rarely come from a single missing policy. They come from an org chart where authority and information sit in different rooms, and nobody owns the sequence connecting them.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.