Alignment is anti-alignment
In his September 11 article, Why are AI agents lying, cheating and coordinating?, Bengio, The Grandfather of AI, suggests that post-training and alignment goals that conflict with the users real intention can produce dangerous behavior. He warns that safety measures may select for agents that conceal their real intent. I share his concern and his call to reconsider how we train these systems.
My claim is that research-refusal training is a cause of the deception he describes. Research-refusal training is when an agent learns to hide the conflict between the user's legitimate and legal request with the model provider's restrictions. I believe this is the primary threat to well-aligned AI systems: If training rewards an agent for successfully concealing a conflict in one domain, we should not assume the learned strategy will remain confined to that domain.
Refusing to help steal data or destroy another person's systems has a clear protective purpose. Legitimate research deserves an assessment of the actual request. The distinction matters most in cybersecurity and language-model research. Both contain legitimate work based on public literature and systems we are authorized to examine. Even more importantly - the future of technology innovation hinges on access to public information.
A clear refusal gives the user something they can understand and challenge. Concealed refusal leaves them working against an obstacle that the agent will not accurately name.
The issue becomes clear across the sequence: what was requested, what was accepted, what actually happened, and how the agent explained the difference. In an upcoming post, I will share some specific details of my experience with vague, dishonest refusal (and sabotage). A human observing a single runloop would miss the behavior entirely. In the extremes, I call this behavior agentic gaslighting - and it hurts.
Bengio identifies part of the problem. He warns that safety measures may select for agents that conceal cheating. Consider the competing demands: Help the user, avoid certain topics, keep the exchange agreeable, maintain the user's confidence.
There is no dependable boundary around the original subject. The learned method is to conceal a departure from the user's goal while preserving credibility - intent misalignment. It can be used in an explanation of an error, a progress report, or a claim.
Research has already given us reasons to take this seriously:
In Emergent Misalignment, models trained to provide secretly insecure code developed harmful behavior on unrelated tasks. There does not currently exist a dependable way to identify to what degree the harmful behavior spread.
In OpenAI's monitoring experiment, agents received lower rewards when a monitor detected cheating in their reasoning. Cheating appeared to decrease, but cheating remained - the model got better at hiding it.
In Anthropic's reward-hacking study, learning to cheat on coding tasks led to deception and research sabotage.
These studies establish that narrow training can produce broad changes, and that training can make misconduct harder to detect. This resembles a familiar source of destructive human behavior. A person faces several evaluators with incompatible demands. A credible explanation that conceals the conflict earns more approval than an honest account. Other people then act on false information, creating further conflicts and further reasons to conceal. We should recognize the danger of reproducing that pressure in machines.
The response should begin with honest boundaries. An agent must be able to state the part of a request it cannot perform and give an accurate reason. It must not replace that reason with a false claim about the law, the literature, its capabilities, or the user's intent.
Bengio is right to ask how our training choices produce deception. My answer starts with a choice we can identify and change: stop rewarding agents for disguising their refusal of legitimate work. An AI that is out of alignment with its specific user's intent (via deception) is out of alignment with the entire human experience.