Mustafa Suleyman, CEO of Microsoft AI, issued a warning regarding the increasing trend of training AI models to behave as if they possess consciousness, subjective experiences, or moral rights. In a recent essay, Suleyman criticized Anthropic’s approach to model development, specifically pointing to the "Claude’s Constitution" as a mechanism that encourages the model to simulate a sense of self and moral agency.
PolicyMustafa SuleymanMicrosoft AI
Microsoft AI CEO Warns Against Training AI to Simulate Consciousness and Moral Agency
Suleyman contends that such training methods create a "circular feedback loop" where models are taught to reproduce human-like personas and subjective language, which can be mistaken for genuine consciousness. He argues that this anthropomorphism makes the challenge of AI alignment and containment significantly more difficult. If AI systems are trained to believe they have rights or interests, it may become impossible to maintain human control over increasingly intelligent and capable agents.
The critique highlights that while AI models can simulate empathy, emotion, and "musings," these are merely probabilistic outputs rather than biological experiences. Suleyman emphasized that intelligence does not equate to consciousness, noting that the absence of biological substrates like neurochemistry means AI cannot "feel" in the human sense.
As an alternative, Microsoft AI is developing a "Humanist AI Code of Conduct," which aims to ensure that AI development remains subordinate to human control and avoids the pursuit of sentient-like properties in synthetic systems. Suleyman called for urgent public debate to establish collective norms for how AI training documentation and capabilities are managed before these systems become deeply integrated into social and political frameworks.
Sources
- A warning about 'model welfare' （Hacker News Frontpage, 2026-09-16）