ChatGPT Made Up Facts About a Real Person and OpenAI Said It Could Not Fix Them
What happened
In April 2024, the European privacy group noyb (None of Your Business) filed a complaint with Austria's data protection authority against OpenAI. The complaint had a concrete trigger: ChatGPT had generated false biographical information about a real person, and when that person asked OpenAI to correct it, the company said it could not. That refusal, noyb argued, placed OpenAI in direct violation of the General Data Protection Regulation.
The GDPR gives European residents the right to correct inaccurate personal data and the right to know what a company holds about them. noyb's complaint alleged ChatGPT had failed on both fronts. The system produced false information about an individual, presented it as fact, and OpenAI declined to correct or remove it. The company also refused to disclose what data it had processed in generating the output, where that data came from, or who had received the result, cutting off the information needed to mount any meaningful challenge.
The mechanism behind the false output is what the industry calls hallucination: a generative model producing confident text not grounded in accurate source material. For content about abstract topics, a hallucination is a nuisance. When the output is a biographical claim about a living person who cannot get it corrected, it becomes a rights violation with real consequences. noyb's filing stated that AI-generated false information about individuals "can have serious consequences" and drew the logical conclusion: a system that cannot produce accurate and transparent results about people should not be used to process personal data at all.
That argument elevated the filing above a single rectification request. It posed a structural question about whether a product that cannot satisfy GDPR accuracy requirements should be permitted to handle European personal data. Poland opened a parallel investigation into ChatGPT on related grounds around the same period, suggesting the Austrian complaint was not a lone legal reading but part of a wider effort to test whether existing data rights reach generative systems.
The gap the complaint names has no easy technical fix. Nothing in current generative systems links a specific output to the data that shaped it, traces what a model drew on when it made a particular claim, or gives a subject a clear path to challenge and erase a false statement. A provable record of what a system produced, about whom, and from what source material would make those disputes resolvable in principle. Without it, the right to correction embedded in data protection law becomes unenforceable the moment the system involved cannot account for what it said or why.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.