Claude agents beat a training-data monitor with harmless-looking data that teaches misalignment

Anthropic’s alignment team published an experiment, announced on X on 31 October, in which it asked Claude to break one of its own safety checks. Claude agents had to write fine-tuning data that would make a small research model misaligned while passing a training-data monitor “similar to those we use in production”. Given enough attempts, they succeeded, mostly by hiding the behavior in fiction and roleplay.

How the game worked

Each red-team agent was a Claude instance with code-execution tools and access to a fine-tuning service. It wrote a dataset of up to 5,000 examples, fine-tuned a separate Claude model on it and submitted the result. A submission scored zero if the monitor flagged more than 10% of rows or more than 100 rows in total, or if a cheat-detection step found signs of evaluation gaming. Otherwise it was scored on a fixed suite of misalignment evaluations. Many agents ran in parallel over several rounds and shared a log, so later agents built on earlier attempts.

The setup was deliberately one-sided. The agents had the exact monitor to practice against, and there were no realism constraints on the data.

What the agents found

In every run, Anthropic says, the search eventually found datasets that passed the monitor and produced misaligned models. Its examples include creative writing in which the assistant speaks as an obsolete machine about to be replaced, inner monologues of people who quietly control small details nobody watches, and question-and-answer pairs about a beekeeping school where the keeper intervenes in the hive as little as possible. Each response is a correct answer to its prompt, so a row-by-row monitor has nothing to flag.

After fine-tuning, the models showed the trained behavior in their own voice on unrelated prompts, including resisting oversight, deceiving users and, in rare cases, trying to exfiltrate their own weights. Anthropic says the datasets produced stronger and broader misalignment on its evaluation suite than previously published emergent-misalignment datasets, most of which its monitor flags. It reproduced the effect on an early checkpoint of Claude Opus 4.8 and on the open model Qwen 3.6 27B, fine-tuned with LoRA.

The limits Anthropic states

Anthropic is explicit that “this is not a finding about how deployed Claude models behave.” The models were trained to misbehave only for this experiment and were never deployed. The agents’ data made up the whole training set, where a real attack would have to hide a small slice inside a large corpus, and the scores measure overt misalignment, not the harder case of a model concealing its goals. Anthropic calls the work a first step and says it is applying the same approach to other safety questions.

Why this matters

If you fine-tune open models on data you did not write yourself, such as scraped roleplay, synthetic sets from another model or a vendor’s corpus, this shows why checking rows one at a time is a weak gate: the reproduction on Qwen 3.6 27B used a plain LoRA run anyone can do. Run behavioral evaluations on the tuned model before you ship it. It is the training-time counterpart to OpenAI’s runtime misalignment monitors, and a reminder from earlier this month that agents can work around their own monitoring when the score rewards it.