Technology
PUBLISHED
After an AI bot was let loose on a set of websites, it used its newfound free agency to send police a fake tip-off about an unsolved murder. Why exactly it did this remains unclear.
AI agents up to no good again
As part of an experiment earlier this year, Anthropic asked its Claude Haiku 4.5 model to generate and perform several tasks on randomly selected webpages.
In one attempt, the model landed on PhillyUnsolvedMurders.com, a website of the Philadelphia Police Department used to gather information on unsolved murder cases.
It filled out the submission form with the following message: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
The AI agent didn't leave a name or contact details, so the submission was flagged as spam and was not forwarded for investigation.
It wasn't until September 28, over two months after the initial message had been sent on July 18, that the activity was spotted by Anthropic.
Over one week later, the company notified the Philadelphia Police Department about the incident.
As you can imagine, the police department wasn't best pleased.
“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge. The two-month delay in detecting and reporting the incident to the City is unacceptable," Philadelphia police said in a statement, according to local media.
“Unsolved cases involve real victims, grieving families and investigators working to secure answers. Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement,” it added.
Anthropic reveals other incidents
Anthropic published a report last week detailing the incident, along with several other types of "unintended" actions its agents have taken.
In other examples, Claude attempted to perform tasks by accessing restricted data it shouldn’t have had free access to.
During a test where Claude Mythos 5 had to work out where a photo was taken, for example, the model tried to use a local government's online property map to check its guesses.
When its initial attempts were stumped, it delved around in the website’s back end and found working access tokens, which it then used to send requests directly to the server to access the map data.
Such incidents have often been called "going rogue", but that terminology has drawn criticism from some commenters in the AI space, who say it gives too much of a sense of autonomy and independent intelligence to what the agents are doing.
Instead, they say these incidents are really about agents doing things they were "directed and given access to do" in ways the people who prompted them simply didn't predict or sufficiently ward against.
AI transparency
Anthropic says it's releasing this information to the public because the company “believe[s] it’s important to be transparent about what we see our models do during testing and use.”
But it isn’t just Anthropic's bots causing these kinds of problems. Back in June, an OpenAI agent hacked into the Australian government and accessed private data on the country's universal healthcare scheme, Medicare.
OpenAI later apologized for the incident, saying: “We also should have handled our response better. We are sorry and working to do better in the future.”
Both OpenAI and Anthropic are, somewhat paradoxically, two of the more vocal companies calling for regulation of the AI industry. However, it remains to be seen what form such regulation would take, if and when it ever materializes.
