Urgent AI safety warnings emerge amidst intensifying US-China technology competition

The global pursuit of advanced artificial intelligence is facing a critical juncture, marked by escalating geopolitical competition and stark warnings from leading experts about the technology’s rapid development outpacing human control. As the United States and China vie for supremacy in AI, a growing chorus of voices within the industry is advocating for a deliberate slowdown in progress to ensure robust safety mechanisms are in place before capabilities become too formidable.

This dual challenge highlights a complex dilemma for policymakers and developers alike: balancing the imperative of national technological leadership with the urgent need to mitigate potential catastrophic risks. The debate has intensified following recent high-profile departures and public statements from figures at the forefront of AI research, signaling a rare consensus among rivals on the need for caution.

Industry leaders call for a controlled pace

Concerns about the unchecked acceleration of AI development gained significant traction after Jacob Coxon, a researcher with experience at OpenAI and Anthropic, resigned from Anthropic, cautioning that fierce competition among leading laboratories is pushing systems forward faster than researchers can reliably ensure human oversight. Coxon reportedly shared with WIRED that colleagues described the coming one to two years as a “crunch time for humanity,” underscoring the perceived urgency of the situation.

Days later, Dario Amodei, CEO of Anthropic, elaborated on these concerns in an essay titled “We Must Pace the Frontier.” Amodei argued for a purposeful deceleration in the rate at which AI model capabilities are enhanced, allowing safeguards to evolve concurrently. His proposals included:

• Embedding independent evaluators within frontier AI development laboratories.
• Establishing coordinated safety standards among democratic nations.
• Developing international agreements to govern the most dangerous AI capabilities.

While emphasizing the need for caution, Amodei clarified that his call was not for a complete halt to AI development, but rather a more measured approach.

Broad support for safety measures among tech giants

Amodei’s advocacy for a more deliberate pace quickly garnered support from unexpected corners of the tech industry, illustrating a rare convergence of views among rival entities. Sam Altman, CEO of OpenAI, publicly endorsed the proposal for independent evaluators, signaling a shared understanding of the risks involved. Elon Musk, a prominent figure in AI and technology, simply affirmed, “Dario is right,” echoing the sentiment. Demis Hassabis of Google DeepMind also voiced support for this direction, while Microsoft CEO Satya Nadella underscored the importance of ensuring that superintelligence serves humanity and remains under human control, deeming any other path unworthy of pursuit.

However, this widespread alarm has also faced scrutiny. Reports of Anthropic preparing for an initial public offering (IPO) that could value the company at nearly $2 trillion have led some to question whether the rhetoric surrounding AI’s extraordinary power and potential dangers might also serve to enhance its perceived value. Jensen Huang, CEO of Nvidia, has previously accused parts of the industry of fueling cybersecurity fears while simultaneously developing products to address them, suggesting that incentives warrant careful examination. Nevertheless, Sam Altman’s decision to postpone OpenAI’s IPO until 2027 due to safety concerns provides a counterpoint, challenging the notion that these warnings are solely driven by financial motives.

US government navigates speed versus safety in AI race

The United States cannot consider these warnings in isolation; the geopolitical landscape, particularly with China, significantly alters the equation. Senior officials have consistently highlighted the comprehensive nature of the competition, extending beyond mere chatbot development to encompass critical areas such as advanced chips, energy infrastructure, data centers, cyber operations, military capabilities, scientific breakthroughs, and global influence. This strategic bind was succinctly articulated by Treasury Secretary Scott Bessent, who stated, “We can’t pause. You can’t, because the Chinese won’t pause.”

Former President Donald Trump, scheduled to host Chinese President Xi Jinping in Washington, has also weighed in on the issue. While affirming that AI would be a major topic of their September 24 discussions and insisting on America’s leadership in the field, Trump dismissed the warnings about slowing AI development as exaggerated. He suggested that unnamed critics were hyping unlikely scenarios, stating, “We’re leading China in AI. Whoever wins AI wins,” while also acknowledging that Washington could still implement guardrails. The administration’s approach under Trump’s Genesis Mission involves linking federal scientific data and supercomputing to AI research across more than 15 agencies, accelerating AI adoption while mandating that systems remain “robust, steerable, controllable,” and subject to human accountability. A separate executive order also provides for voluntary pre-release federal cybersecurity evaluations of frontier models.

Congressional debate and international challenges

The debate over AI governance has also taken root on Capitol Hill, though a comprehensive statutory framework remains elusive. House Speaker Mike Johnson, a Republican from Louisiana, has rejected calls for an emergency moratorium on AI development, cautioning that hasty regulation could allow China to close the technological gap. Echoing a free-market perspective, former White House AI adviser David Sacks argued that laboratories wishing to slow down should do so independently, without leveraging federal intervention to impose similar restrictions on smaller competitors.

Conversely, other lawmakers, such as Representative Ted Lieu, have proposed more stringent measures, ranging from outright bans on superintelligence to requirements for developers to test and disclose catastrophic risks. While there is a broad consensus in Washington that significant developments are unfolding, agreement on a unified course of action remains divided. The international dimension further complicates matters, particularly concerning China. On September 8, U.S. cyber and law enforcement officials accused Chinese AI companies, including DeepSeek, Moonshot AI, and Alibaba, of systematically acquiring American AI technology through “industrial-scale” distillation. Beijing refuted these allegations, characterizing distillation as a common industry practice. This ongoing dispute underscores the inherent dangers of unilateral American restraint, as Washington can regulate its domestic laboratories but cannot compel Beijing to adhere to identical limitations. China’s Foreign Ministry explicitly rejected Amodei’s proposal, denouncing what it termed “fearmongering” and warning that confrontation would disrupt global AI governance. This highlights the delicate balance for the U.S., which seeks Chinese cooperation on slowing dangerous capabilities while simultaneously restricting Beijing’s access to critical chips and model capabilities that could advance its own AI efforts.