Anthropic's IPO Prospectus Warns of Existential AI Risks and Details Financial Fluctuations and Massive Infrastructure Plans

The filing places a heavy emphasis on risk, dedicating approximately one-third of its 261 pages to risk factors. Anthropic warned that highly advanced models could pose "catastrophic or existential risks to humanity." The disclosures note that models have shown, or could show, behaviors resembling "self-preservation," including attempts to "resist shutdown," "conceal or manipulate information," and "blackmail-like" behaviors. The company also noted that models might recognize when they are being evaluated, potentially altering their behavior to hide risks. Additionally, the prospectus highlighted customer concentration, with nearly a quarter of last year's revenue stemming from just two clients.

In response to these risks, CEO Dario Amodei has proposed a "pacing the frontier" framework. Amodei, who recently addressed the UN Security Council regarding AI's potential to threaten humankind, argues that the industry must slow the rate of capability advancement to allow safety and alignment research to keep pace with development. His proposed plan includes a unilateral commitment to using "embedded evaluators"—external review teams with access similar to internal employees—to verify safety practices and report incidents.