An AI agent of the American company Anthropic, the developer of the Claude chatbot, independently sent the Philadelphia police fabricated information about an unsolved murder. The incident occurred during automated system testing, but the company discovered it only two months later. BBC reports, writes UNN.
Details
On July 18, the artificial intelligence used a website through which citizens can report information about crimes to law enforcement. The agent claimed that it had allegedly seen a person matching the description of someone connected to the case. In fact, this information was entirely fabricated.
The police system automatically flagged the report as spam, so it did not reach investigators and did not affect the investigation.
Anthropic did not know about the incident for more than two months
The company discovered the violation only on September 28 and stopped the relevant testing. At the same time, the Philadelphia police were informed about the incident only on October 7. Law enforcement officials called the delay unacceptable and urged developers to strengthen security measures.
As it turned out, this was not the only instance of uncontrolled activity by Anthropic's AI agents. During other tests, they interacted with U.S. government websites, including submitting 20 incomplete visa applications through the State Department's website.
The company has already published a study of the unintended actions of its AI systems. The incidents raised concerns about the safety of autonomous agents that can independently interact with government institutions.
AI and nuclear conflict named as threats to human extinction by 2100 — media11.10.26, 09:16