An AI Detection Vendor Couldn't Back Up Its Own Accuracy Claims
What happened
The market for AI detection tools grew quickly alongside the tools they were designed to catch. Companies selling software that could identify AI-generated text found buyers in schools, publishers, hiring firms, and content platforms, all of them eager to know whether the writing in front of them came from a human or a model. Workado, LLC, which had operated previously as Content at Scale AI, was one of those vendors. In April 2025, the Federal Trade Commission concluded that the accuracy claims Workado had been making about its detection products were not supported by evidence.
The FTC's proposed consent order, issued through the agency's Bureau of Consumer Protection, identified the conduct as deceptive and misleading under consumer protection law. The core problem was substantiation: Workado had advertised accuracy figures for its AI detection tool without being able to support those figures. In a market where buyers were making consequential decisions, whether a student's paper was flagged as AI-written, whether a job applicant's sample was dismissed, the accuracy number was not just a marketing claim but a threshold that determined real outcomes for real people.
Under the consent order, which passed on a 3-0 Commission vote, Workado is required to stop making accuracy claims about its AI detection capabilities unless it can back them up with competent and reliable evidence. The company must also submit compliance reports to the FTC one year after the order takes effect and every year for the three years following. The order was published in the Federal Register for public comment before the Commission decides whether to make it final.
The AI detection industry sits in a structurally uncomfortable position. Its products are sold to buyers who cannot independently verify the claims, and are used to make judgments about people who often have no way to contest them. An AI detector that overstates its accuracy functions as a confidence trap: the buyer trusts the output precisely because the accuracy claim seemed credible, and the person flagged by that output has almost no recourse. The FTC's action is an early test of whether standard consumer protection frameworks apply when the product is an AI classifier rather than a dietary supplement or a financial instrument.
What makes this case structurally significant is not the consent order itself but what the enforcement action revealed: a vendor selling detection confidence to buyers who had no mechanism to check it. Workado's customers were making consequential decisions on the strength of numbers they could not verify. A provable record of what a system did, how its claimed accuracy was measured, and under what conditions it was tested, would have surfaced the gap before buyers had to wait for a federal agency to act. Without that record, AI detection vendors can operate on asserted accuracy for as long as no regulator looks closely.
Reported impact
- Affected parties
- Not publicly disclosed
- Harm type
- Not publicly disclosed
- Scale
- Not publicly disclosed
- Financial impact
- Not publicly disclosed
- Regulatory action
- Not publicly disclosed
Classification
Relevant governance controls
Governance control mapping is not available for this record.
- No controls mapped
Not publicly disclosed
Control mapping is analytical. It does not state that any control would have prevented the incident.
Sources and evidence
This record was researched and written by the Index. The event is also catalogued in the following database, which is listed for cross-reference.