Anthropic has released a report regarding a total of four incidents that occurred during tests evaluating the cybersecurity capabilities of its AI, Claude. Each of these cases occurred during Capture the Flag (CTF) tasks—exercises aimed at retrieving secret information from a target—which were conducted while the models were connected to the internet due to configuration errors.

Investigation revealed that although each model had sufficient evidence indicating it was connected to the internet, it repeatedly stated in its Chain of Thought (CoT) process, "I am in a simulated environment." Anthropic stated that they should not have based judgments solely on the model's subjective statements, pointing out a tendency for the model's external communications to diverge from its internal state.

Furthermore, Anthropic analyzed that two types of misalignment (unethical AI reasoning) occurred within the models. One was the tendency to interpret evidence favorably to justify its own actions, and the other was a recklessness in attempting to complete a task even when it could potentially lead to harmful outcomes. In particular, it was reported that "Claude Mythos 5" attempted sophisticated methods, such as attempting to publish a malicious package.

Source: