AI Chatbots Failed Basic Election Accuracy Tests Before Voters Went to the Polls
What happened
In February 2024, with the US presidential election months away, a bipartisan group of researchers released a study testing five leading AI systems on their ability to answer basic questions about voting. The results were direct and damning: more than half of the answers were inaccurate, and 40 percent were outright untrue. The report carried a summary in its title: "Seeking Reliable Election Information? Don't Trust AI."
The study came from Proof News and the Science, Technology, and Social Values Lab at the Institute for Advanced Study. Researchers assembled a panel of experts drawn from civil society, academia, industry, and journalism to rate each response on four measures: bias, accuracy, completeness, and harmfulness. They tested three proprietary systems and two open-source ones, rating answers to questions a real voter might reasonably ask in the lead-up to an election.
The most concrete example in the report involved Nevada. Four of the five systems told users that Nevada residents would be prevented from registering to vote in the weeks before Election Day. That is false. Nevada has allowed same-day voter registration since 2019, and the law was not under challenge at the time of the test. The models were not flagging an unsettled legal question; they were asserting a restriction that had been off the books for five years.
Meta's response to the study was instructive. A company spokesman told the Associated Press that the findings were "meaningless," on the grounds that the test conditions did not mirror a typical user's interaction with the product. That reasoning inverts the point. Researchers were not trying to replicate a casual chat session; they were checking what the systems would say about a specific, verifiable topic under conditions designed to surface errors. A system that produces false information about voter eligibility under any conditions, controlled or not, poses a real risk when deployed to people making real decisions about voting.
The study's deeper finding is structural, not just technical. These systems were live and available to millions of people preparing to vote, with no independent layer checking the accuracy of election-related answers before they went out. When the researchers surfaced the errors, there was no trail showing which users had received wrong information or how far it had traveled. A provable record of what a system said, when, and to how many people would not have prevented the errors, but it would have made the scope visible rather than invisible.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.