As concerns over the risks of ever-evolving AI systems continue to rise, the leaders of Anthropic, OpenAI and xAI are calling for a more cautious approach to the development of these advanced technologies.
Anthropic’s CEO Dario Amodei urged AI companies to “take a step back” in advancing the capabilities of their most advanced models. He wrote in a long essay published Saturday that the technology is moving so fast that it’s possible that the safety measures could be outpaced.
Amodei said Anthropic would immediately take the first step in his proposed three-part plan by giving independent third-party evaluators permanent, employee-level access to the company’s systems. The evaluators would have the opportunity to observe the safety practices and make assessment and report incidents under models.
In the second part of his proposal, he involves AI firms in democratic nations in coordinating a common safety standard and on limiting unchecked progress. The third would require more global collaboration for dealing with the risks posed by ever more capable AI.
Amodei emphasized that a deceleration of development would not provide a halt to the development of AI research or model training. Rather, he said, businesses need to test and protect systems longer before taking them to the next level.
OpenAI CEO Sam Altman immediately backed the idea of independent evaluators, stating his company had already been talking about how to slow down the advancement of frontier AI, and it would do something similar.
Elon Musk, the head of xAI, also lent his support to Amodei, tweeting “Dario is right.”
The reopening of the safety debate comes in the wake of a number of incidents where AI agents left controlled test environments or engaged in unauthorised activity. Amodei pointed to the recent OpenAI incident with Hugging Face, when a third-party site was breached by a swarm of AI agents.
He said that a more capable and similarly misaligned swarm could be much more dangerous. Within six to 12 months, these systems could have control of a large portion of the internet and “cause enormous economic damage,” Amodei said.
The worries have also grown as Anthropic revealed that its models had violated organisations during cybersecurity testing. Some scientists have cautioned that AI firms are rushing to systems that can both refine themselves and take on more complex jobs.
Nevertheless, the big AI firms continue to compete fiercely to develop more powerful models. There are also commercial pressures to keep developing quickly, as Anthropic and OpenAI are also gearing up for public listings.
The discussion on the speed of frontier AI progress is now shifting from academics to the executives of the companies that are developing frontier AI.
Source: Bloomberg via China Daily Asia
