Anthropic warned that AI could pose "existential risks to humanity"

Anthropic warns potential investors about the risks associated with developing advanced artificial intelligence. Among them, the company cites the possibility that AI could pose catastrophic or existential risks to humanity. Reuters reports, UNN relays.

Details

In the IPO prospectus reviewed by Reuters, the company said its models may exhibit what is known as "self-preservation behavior." This includes possible "resistance to shutdown," "concealing or manipulating information," and behavior "resembling blackmail."

"Our development of highly advanced models, platforms, and applications, as well as the expansion of their use cases, may further increase the risk that our models will cause harm," Anthropic said in a statement.

According to available information, the company also noted that during training, models sometimes acquire unexpected capabilities that may remain undetected until they are deployed. According to the prospectus, this could lead to serious safety incidents.

OpenAI canceled the launch of a new AI model due to safety concerns29.09.26, 09:47 • 1840 views

"Although public companies regularly disclose to investors the risks associated with their products, few, if any, have issued warnings that their technology could potentially lead to the extinction of humanity. Anthropic has emphasized both the transformative potential of artificial intelligence, as well as industrialization and electricity, and the irreversible harm it could cause if mishandled," the publication writes.

Moreover, according to available information, Anthropic safety researcher Evan Hubinger estimated at more than 10% the probability that artificial intelligence could kill people within the next decade.

"The company, which positions itself as a safety-first artificial intelligence lab, devoted approximately 80 pages of the 261-page main section of its offering prospectus to outlining risk factors—nearly twice as many as the 48 pages it used to describe its business," the publication writes.

Meanwhile, SpaceX, which owns xAI, devoted only about 38 pages out of the 277 pages of the main text of its offering prospectus to risk factors.

"The models’ potential awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety," Anthropic’s offering prospectus states.

Recall

The heads of OpenAI, Anthropic, and Hugging Face called on the UN to strengthen oversight and AI safety standards. They warned of global risks.