Fears over the safety of AI systems — and their potential to wipe out humanity — gained new, viral traction this week. Evan Hubinger, an alignment lead at Anthropic, said on X that he thinks there is more than a 10% chance that AI could kill all humans within the next decade, after a colleague quit over safety fears. More warnings from researchers at both Anthropic and OpenAI followed. Cue a social media frenzy. But it was in Hubinger’s reply to his own post that revealed where exactly his concerns lay. “What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought,” he said. Recursive self-improvement, or RSI, is when AI itself helps improve the process of building new models, potentially leading to spiralling capability as better systems build better systems and so on.

Full report : AI researchers are warning that faster AI self-improvement could eventually make advanced systems harder for humans to control.