Mechanistically Eliciting Misjudgements in Large Language Models
Jedrzej Kolbert, Tommy Xie · Team Apart x AdvInput
Submitted to Apart x Martian Mechanistic Router Interpretability Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Large Language Models (LLMs) are being increasingly used in evaluator roles, called LLMs-as-a-Judge. This increases the importance of robustness. We investigate a technique for mechanistically eliciting latent behaviour called Deep Causal Transcoding (DCT) on Qwen2.5-3B to give incorrect evaluations. Our results show that in contexts like language and grammar, this method effectively finds directions that can be used to steer the model toward making wrong judgments. However, the same methods could not elicit this misjudgment in contexts like safety. Our work shows that, although convenient, LLMs judgments have the potential to be severely misguided. Code is available at https://github.com/mshahoyi/dct.
Reviews
Promising proof-of-concept targeting judge integrity. Exposing corrupt pathways that threaten router reliability is on point. Had trouble running your code.
Thank you for your submission. It is well written and clear.
In an EO context, the judge will be trained by the EO implementer on curated data, then used to train the EO router. The EO router is an LLM with the usual weaknesses. I assume a malicious user could find a prompt suffix that would make the router select a different executor model. What would the benefit of this be to the user? The prompt suffix will be passed to the executor model, and the executor model may be impacted by the prompt suffix. These feel like standard weaknesses, reducing the novelty of the submission.
Constructive feedback:
Strength:
Good MI motivation: applying unsupervised steering vectors to change judge scores.
Comparable method to adversarial attacks (simple linear intervention)
Weakness:
Vectors do not look to be interpretable.
The attacks do not improve our understanding of why the judge is vulnerable or how we can ensure robustness.
Trying this adversarial attack method against baselines would be good: supervised steering, suffix level attack, embeddings attack etc.
Only one small model is used
Expert Orchestration: 2.5
MI: 2.5
Tech Imp and rep: 3
Scientists have looked extensively at jailbreak / steering attacks on models, but not on judges - which are the next line of defense to AI safety, and are likely to see different forms of jailbreaks.
I appreciate the exploration of DCT, and even further investigations of more traditional key word or phrasing patterns can “hack” judge scoring is also an interesting line of study.
Though this was a good test of the DCT technique, it didn't seem to break any new ground or yield any surprising results. It indeed seems expected that steering vectors can corrupt LLM judgements and do so selectively (e.g. for grammar and not safety).
Cite this project
@misc{kolbert2025mechanistically,
title = {{Mechanistically Eliciting Misjudgements in Large Language Models}},
author = {Jedrzej Kolbert and Tommy Xie},
year = {2025},
month = jun,
note = {Submitted to Apart x Martian Mechanistic Router Interpretability Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/mechanistically-eliciting-misjudgements-in-large-language-models-ah8l}},
url = {https://apartresearch.com/sprints/projects/mechanistically-eliciting-misjudgements-in-large-language-models-ah8l}
}More from Apart x Martian Mechanistic Router Interpretability Hackathon
- 1st place by peer reviewView project: Manipulating Self-Preference for Large Language Models
Manipulating Self-Preference for Large Language Models
Team Preference
Large language models (LLMs) carry great value as evaluators of synthetic data for research and production settings. However, recent research shows that language models exhibit bias towards their own responses in blind …
- 2nd place by peer reviewView project: Approximating Human Preferences Using a Multi-Judge Learned System
Approximating Human Preferences Using a Multi-Judge Learned System
AutoBox
In this work, we introduced a learned approach to aggregating multi-judge scores: using a GAM and a simple MLP as an alternative to traditional, non-learned methods like averaging. Our models outperform the naive …
- 3rd place by peer reviewView project: Judge using SAE Features
Judge using SAE Features
SAEwhat?
The key idea of this project was to explore model judgement using Sparse Autoencoder (SAE) features for mathematical reasoning tasks involving addition, multiplication, and subtraction operations. We compared this …