Anthropic co-founder on AI safety and power

Anthropic co-founder Jack Clark told the BBC in mid-September that a mandatory, third-party-verifiable 'kill switch' allowing AI systems to be shut down if they become dangerous may need to become a legal requirement for AI companies, saying 'most labs have different ways of being able to pull the plug' but that policymakers may eventually need to enforce having one. Clark's comments followed Anthropic CEO Dario Amodei's weekend call for AI development to slow and be more closely monitored, and came amid a wave of public warnings from AI figures: Anthropic scientist Evan Hubinger estimated the chance of human extinction from AI within a decade at over 10%, a figure Nobel laureate Geoffrey Hinton called 'not unreasonable,' while Clark himself said 'we are rolling dice with immense risks' by leaving the industry effectively unregulated. US lawmakers had introduced a 'Kill Switch Act' that would require companies to maintain a shutdown mechanism for AI tools and empower government agencies to demand one be triggered, but President Trump dismissed AI-extinction fears as a 'HOAX' and accused unspecified critics of running a 'SICK conspiracy' against AI and data centers; the UK government separately rejected mandating a kill switch, arguing it 'would not prevent them being developed or misused elsewhere.'

OpenAI CEO Sam Altman, in a series of X posts the same week, warned that AI progress could go 'very badly' either through humanity losing control of AI systems or through power over extraordinarily capable AI becoming concentrated in one company or country, calling for a 'narrow middle path' and consistent federal safety standards, while insisting 'pacing' AI development did not mean stopping it. Former President Barack Obama also weighed in publicly, welcoming frontier labs' recent moves to slow development as a 'good and necessary first step' and arguing that decisions about AI's trajectory 'should be made not just by the companies involved, but by all of us.' Anthropic, maker of the Claude chatbot, was reported to be preparing a potentially record-setting IPO, while OpenAI's Altman said his company would not pursue a similar public listing this year given the ongoing AI-safety debate.