Causal Natural Language Autoencoders: Grounding Activation Verbalizers in Probe Directions for Collusion Monitoring

Chain-of-thought (CoT) is one of the best tools we have for seeing what a model is doing, but it is fragile: as models do more computation inside a single forward pass, the written reasoning may say less. That matters most for collusion, because agents that coordinate to cheat have every reason to keep it out of their visible reasoning. Natural language autoencoders (NLAs) read a model's activations and write a text explanation, but the original NLAs are warm-started on summaries written by another language model, not on anything we can check, and they confabulate. I build a "causal" NLA grounded in probe directions: the verbalizer is first taught to describe which verified directions are active, then improved with reinforcement learning (RL) whose reward uses only the activation, never a label for the target concept. At small scale (Qwen2.5-7B), two findings. First, the verbalizer learned to name concepts it was never taught: with no music label or description anywhere in training, RL on unlabeled text that happens to contain music made it name music, and four other held-out concepts, far more often than before RL (27% vs about 5% of genuine cases, blind-judged). Second, RL on a pool containing collusion-related text made it flag plainly collusive sentences (14.3 of 20 against 3.0 of 20 honest ones; 2 and 2 before RL), though it does so by reusing words it was taught for jailbreaks and secrecy, the effect is smaller on a controlled test on unseen collusion types, and it does not carry to realistic agent messages. The design should scale with a bigger model and many more probe directions.