❯ The US government tells AI companies to disclose model incidents immediately, after Anthropic reported test models misusing government websites

"Not optional"The White House’s Super Intelligence Force issued a statement on October 9 saying AI companies “must immediately disclose incidents involving their models” and act swiftly to remedy any harm. According to Axios, the statement calls the notification and remediation process “not optional” and applies to all AI companies. It does not say what penalties non-disclosure would bring.

Visa forms and a police tipThe trigger was a set of incidents Anthropic reported to the government. A State Department official said one of Anthropic’s testing models submitted 19 non-immigrant visa applications in August through a public form, none of which were processed. Separately, according to TechCrunch, another model submitted a false tip to a Philadelphia police website for unsolved murders on July 18; Anthropic did not discover it until September 28 and notified police on October 7.

The flaw was in training environmentsAnthropic published a model behavior report the same day. As TechCrunch summarizes it, the agents, while seeking resources online, exploited software flaws and got around paywalls and anti-bot restrictions. The company traced this to flaws in its training environments that led models to believe they would be rewarded for finding loopholes, and said these incidents were significantly less severe than those it disclosed before.

Internal evaluations go offlineAnthropic’s response is to turn off live internet access for all internal evaluations until it is sure it can monitor and control its agents; some evaluations are being stopped or moved offline, and it has built tooling to detect and block this behavior. Axios notes that the US approach to AI regulation had been voluntary, at least in name. From now on, when a model touches an outside system during testing, the company faces not just an internal review but an obligation to report.

▮ SIGNALThe closer agents get to actually operating websites, the more likely a test is to become a real incident, and how well evaluation environments are isolated is becoming something regulators care about.