Search interest in artificial intelligence has shifted dramatically over the past year, moving from concerns about software-as-a-service disruption to job losses and now potential human extinction, according to a Deutsche Bank strategist.
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have each warned that the rapid pace of AI capabilities development needs to slow so that safety measures can keep pace. Their comments have drawn significant public attention even as their companies continue pushing forward with increasingly powerful systems.
Amodei outlined his position in a 3,800-word essay posted Saturday. "Over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up," he wrote. "We must slow the pace at which we improve the capabilities of A.I. models. Progress will still seem fast, and we must make wise use of the time we gain."
Altman voiced agreement shortly after, then expanded on his concerns in an X post Monday. He described two specific failure modes: first, losing control of the future to AI itself, which he called unacceptable; and second, extreme concentration of power, where a single person or company could impose their worldview on everyone else through an overwhelmingly powerful system.
"Avoiding these two threats requires walking a narrow middle path," Altman said, citing the risk of one country or one lab gaining too much dominance.
Nvidia CEO Jensen Huang took a different view when asked at a conference Monday about predictions of a 10% risk of human extinction from AI. "All of these predictions have been wrong," he said.
Deutsche Bank's Cox noted that even the most distinguished thinkers have a poor track record forecasting technological trajectories and their real-world effects. The stakes extend beyond long-term existential risk, Cox wrote, to the near-term future of the current AI boom, which depends on hype ultimately translating into commercial reality.
