AI giants pledge to act over fears tech is developing too fast

Tech bosses Elon Musk and Sam Altman have backed a call from the head of Anthropic to have permanent third-party reviewers inside "frontier" AI companies.

Anthropic chief executive Dario Amodei has written an essay called We Must Pace the Frontier, in which he urges such companies to slow the rate at which they improve the capabilities of AI models.

It comes amid increased fears about the possible misuse of AI, with Anthropic recently revealing that several bad actors had used its Claude AI models for activities including weapons development, cyber operations, surveillance ​and fraud.

👉Listen to This Is Why on your podcast app👈

Mr Amodei has come up with a three-part plan, part of which he said his AI company was unilaterally committed to.

In a post on X in which he commits to the "first of these steps", he writes: "We'll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training."

Mr Musk, the boss of X which has developed it's own 'Grok' AI, replied to the post, writing: "Dario is right."

And OpenAI boss Mr Altman wrote on X: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

"Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon."

Mr Amodei said he was concerned that since this summer, AI has been "advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI".

And he warned that within six to 12 months AI could be capable of leading a "swarm" that could take over "the entire internet" causing "hundreds of billions of dollars in damage", among other risks.

Former prime minister Rishi Sunak wrote on X: "Dario is right to call today for slowing the pace of frontier development so that safety can keep up.

"When the labs themselves are asking for speed limits, governments should listen."

He added that "even as the US and China compete, they urgently need a frank conversation about how to avoid catastrophe".

Evan Hubinger, a top safety researcher at Anthropic, recently said there is a more than 10% chance that "AI could kill all humans" in the next decade.

And researcher Jacob Coxon ⁠resigned from Anthropic amid fears the company and its chief rival OpenAI "are racing straight to self-improving superintelligence and gambling with our lives".

Read more from Sky News:
Boy, 15, found clinging to capsized boat after days at sea
Woman gives birth after using embryo frozen for 22 years

Mr Amodei said he was not calling for stopping model training or technical progress, but making sure companies take enough time to align and ‌safeguard their models, and for third-party evaluators to confirm these steps.

What are his proposed three steps?

• Embedded evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.

This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory "supervisors" embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work.

• Democratic coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.

• Global coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.

(c) Sky News 2026: AI giants pledge to act over fears tech is developing too fast