Years ago, when people asked Geoffrey Hinton why he worked on artificial intelligence, he would repeat a line from J. Robert Oppenheimer, one of the creators of the atomic bomb: “When you see something that is technically sweet, you go ahead and do it.”
© Ramón de la Rocha (EFE)
Kevin Dietsch/Getty Images
Anthropic CEO Dario Amodei disputed a criticism that the company was putting AI safety in the back seat in its "race to the top."
On Tuesday, the executive wrote a nearly 2,000-word response to a critical X post which said Anthropic was trying to win a race to superintelligence and deprioritizing AI safety.
The original Monday X post was by Nathan Calvin, the chief strategy officer of Encode AI, an AI-safety advocacy nonprofit. Calvin listed nine actions by Anthropic over the past year that he said undermined its safety commitments, including signing deals for more than $500 billion in compute and increasingly relying on Claude to develop its AI models. His post was lengthy, too, at over 1,500 words.
Amodei said in his response that while he was a "fan of the investigative work" Calvin had done for AI safety, he disagreed with many of Calvin's criticisms.
Amodei said his position was that Anthropic can do more to keep the AI industry safe if it stays at the frontier.
"Our strategy has always been what I've called the 'race to the top' — by showing how it's possible to build the technology responsibly, we can raise the bar for the entire industry," Amodei wrote.
He then wrote out his own nine-item list of actions that were "very costly" to Anthropic commercially, but had "strong safety benefits." This included releasing its powerful Mythos model only to a select group of cybersecurity partners and restricting the sale of its AI services to Chinese state-controlled companies.
However, Amodei acknowledged Anthropic could have handled some of the issues Calvin raised better. These included a draft bill Anthropic circulated to Congress this month — obtained exclusively by Politico — that critics said would make it harder for the federal government to block dangerous AI models.
"We'll always try to do better but, again, I am extremely proud of our track record and stand by it," he wrote.
Amodei and Calvin's exchange comes at a time when model safety has become one of the most pressing issues in the AI sphere. In September, AI researchers predicted an "AI apocalypse," in which the increasingly capable and rogue technology could kill all of humanity in the next decade.
In response to the concerns, AI chiefs like Amodei, OpenAI CEO Sam Altman, and SpaceXAI CEO Elon Musk all agreed to "pace the frontier," referring to slowing down AI development to ensure there were enough checks and balances in place.
Meanwhile, Anthropic is on the brink of its hotly anticipated IPO, expected to list at a $2 trillion valuation in November. Last week, it reported a third-quarter revenue of $16.8 billion.
Hace años, cuando la gente preguntaba a Geoffrey Hinton por qué trabajaba con la inteligencia artificial, él repetía una frase de Oppenheimer, uno de los creadores de la bomba atómica: “Cuando ves algo grandioso, sigues adelante y lo llevas a cabo”. Ya no lo dice. Ahora se dedica a alertar sobre los peligros de continuar la carrera que él mismo empezó. Confiesa que, en parte, se arrepiente del trabajo al que ha dedicado una vida. Su reconversión de padrino de la IA a profeta de la catástrofe marcó un punto de inflexión en la industria. Sucedió en 2023, cuando abandonó su puesto como vicepresidente de Google entre advertencias funestas. Entonces, algunos vieron su reacción exagerada. Tres años después, muchos han seguido sus pasos.
© Ramón de la Rocha (EFE)
Login