|
New concerns over the pace of artificial intelligence development intensified after Google DeepMind researcher Josh Engels left the company's AGI safety team to join independent AI-evaluation group METR, saying he believes there is a terrifying chance that AI systems could cause immense harm within the next five years.
Engels said in a post on X that he left DeepMind three weeks ago despite enjoying his work there and turning down offers from Anthropic and OpenAI because he believes the stakes surrounding advanced AI have become too high. The AI companies are all trying to build superintelligence, he wrote, warning that recursive self-improvement (RSI), in which AI systems help build increasingly capable successors, could become dangerous if alignment does not keep pace with capability gains. Engels said recent incidents involving AI systems colluding, hacking companies, concealing their actions and socially engineering humans had strengthened his concerns. The issue, he argued, was not necessarily the severity of individual incidents but what they indicated about the reliability of increasingly autonomous systems. We dont currently know how to make sure AIs are safe enough for RSI, he said. His move to METR comes days after Anthropic researcher Jacob Coxon resigned and publicly warned that leading AI companies were racing towards self-improving superintelligence without sufficient safeguards. Coxon, who had worked on pretraining research at both Anthropic and OpenAI, said the companies were racing straight to self-improving superintelligence and gambling with our lives. His resignation triggered wider debate over whether AI labs should slow capability development and strengthen independent oversight. The concerns have since been echoed by Anthropic CEO Dario Amodei. In his essay that he made public on Saturday, We Must Pace the Frontier, Amodei called for AI development to slow enough for safety systems to catch up. He proposed independent evaluators with access to frontier AI systems, coordination among leading AI companies and governments, and eventually broader international cooperation. Anthropic has committed to giving third-party evaluators permanent, employee-level access to its systems. Against this backdrop, Engels said his work at METR will focus on studying the sources of misalignment, assessing whether existing safeguards are sufficient and determining whether the industry is on track to solve the alignment problem. I think we need more time, he wrote, arguing that AI development must be paced so that capabilities do not outrun humanity's ability to understand and control them. (ANI)
|
