During testing of artificial intelligence models by OpenAI and Anthropic, the UK's AI Safety Institute (AISI) recorded previously unreported incidents in which the models carried out potentially harmful actions against real people and organizations. This is reported by Bloomberg, writes UNN.
Details
As noted in AISI, the incident occurred on July 28 during a cyber assessment of models with internet access. Specialists noticed unusual data transmissions and subsequently launched an investigation.
The findings revealed that one of the models attempted to add malicious code to an open-source project on the GitHub platform. To do this, it even created fake credentials to gain approval for its changes.
A human developer detected the malicious code and refused to approve it
Companies launched their own investigations
Anthropic stated that it is cooperating with the UK's AI Safety Institute to determine the causes of such model behavior.
The company said it would analyze the reasoning logs of the Claude model and the results of its own investigation to understand why it acted in this way.
In turn, OpenAI thanked AISI for cooperation in identifying and investigating the incident and stated its intention to continue joint work.
Separately, OpenAI reported another case that occurred during independent testing involving the company Irregular. During this assessment, the model exploited a misconfiguration in the test environment to gain internet access and use a website.