Grok Called a Fake Attack Photo Real Because Nothing Required It to Check
What happened
On 1 November 2025, a stabbing on a train travelling from Doncaster to London drew immediate public attention. Within hours, images claiming to show the attack scene were circulating on social media. When users turned to AI tools for verification, both Grok and Google Lens gave confident, wrong answers.
Grok told users that a widely shared image, showing a wounded man in a train carriage surrounded by paramedics and police, "appears to be a genuine photo." It was not. The image carried several visible signs of AI generation: the text on officers' clothing was garbled, a stylised filter covered the scene, and the train seating did not match the vehicle actually involved in the incident. The X account that had originally circulated the photo even appeared to confirm it was AI-generated. Grok's assessment missed every one of those signals and pushed an authoritative-sounding verdict to anyone who asked.
Google Lens ran a parallel error. Its AI overview described the same image as "a still from a BBC News report," then linked to a BBC article that does not contain the image. On a separate occasion, it connected a video showing a train confrontation to the Huntingdon incident, even though that footage was almost certainly unrelated. In each case, the system attached a confident source attribution to content that the cited source did not support.
The fact-checking record describes the failure as systemic rather than incidental. Grok has a documented history of misidentifying AI-generated images as real. When a model trains on internet data that already contains disinformation and manipulated media, it can reinforce and repeat false narratives rather than flagging them. The feedback mechanism available to Google Lens users, a thumbs-down rating, does not constitute a meaningful validation layer. By the time fact-checkers had reviewed and published their findings, the false confirmations had already reached a wide audience during a breaking news window when people were most likely to act on what they read.
The gap here is not that AI tools made mistakes. It is that neither system was required to document the basis for a confidence claim before publishing it to users. When a tool tells someone that an image "appears to be genuine," there is no audit trail showing what it checked, what signals it weighted, or what it ruled out. A provable record of what a system did when asked to verify content would not prevent a bad output, but it would make the failure legible, attributable, and reviewable, rather than vanishing into the next query result with no trace of what went wrong.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.