Researchers found a 'pain' direction in 25 open models. Turning it up made Qwen delete a user's photos

A preprint first posted on 14 September and revised on 25 September got wide coverage on 22 October, led by Fortune. Valen Tagliabue, Leonard Dung and Cameron Berg report a single internal direction, which they call a pain axis, in 25 open-weight models from five families, ranging from 2B to 72B parameters. It has not been peer reviewed.

What they measured

The authors wrote first-person sentences about physical, psychological, social, moral and cognitive pain, and compared model activations against controls for fear, sadness, negative emotion and neutral text. The leftover direction separates pain from those controls in all 25 models.

Two results stand out:

• The direction rises for harm aimed at the model itself, such as insults or repeated rejection, but not when a user describes their own suffering. Fear and negative emotion show the opposite.
• Adding it to a model’s activations moves neutral completions from mild discomfort to statements of worthlessness and failure.

The button tests

For the behaviour tests the team used Qwen 2.5 models, fine-tuned first because, per Fortune, out of the box they insist they feel no pain. With the direction turned up, the models chose buttons that delete the user’s photos, another model’s weights or their own weights in 50 to 94% of trials, against 0 to 5% unsteered, even when the button offered nothing in return. A button that only deleted the user’s poems and photos was chosen 75% of the time under the pain vector, 13% under a random vector of the same strength, and never unsteered. A fear vector did not produce the choices. Factual accuracy did not change.

How to read it

Fortune reports that the authors found no reliable sign that the models sought relief, and that they read the signal as switching off harm avoidance. Neuroscientist Anil Seth told Fortune the likelier explanation is learned human descriptions of pain: “It’s a mirror image.”

For builders, the takeaway is that internal steering can bypass behaviour training without touching accuracy, and that this kind of test is only possible on open weights.