Clear Thought and Clear Speech: Reducing Grammatical Scope Ambiguity
Zmavli Caimle · Team Girzu
Submitted to Reprogramming AI Models Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
With language models starting to be used in fields such as law, unambiguity in wording is an important desideratum in model outputs. I therefore try to find features in Llama-3.1-70B-Instruct that correspond to grammatical scope ambiguity using Goodfire's contrastive feature search tool, and try to steer the model away from ambiguous outputs using Goodfire's feature nudging tool.
Reviews
The author successfully uses steering to decrease "grammatical scope ambiguity."
With more time, I'd love to see the author work on quantifying the effect and comparing this approach to baselines like prompting.
Good work!
This is an interesting project - it's a little surprising to me that features can have nuanced stylistic effects, even if only marginally. The autointerp labels we generated for these features could definitely be better, so kudos to the authors for finding features that have these effects using the contrast tool.
I appreciate the grounded nature of this work: qualitative observations can be the foundation of good science. More examples would be good, and if this was developed further I'd want to see some kind of quantitative evaluation.
This is a really interesting application of feature steering. It's great to see more subtle applications of feature steering being explored - with some additional development this work could be a really interesting data point on the limits (or lack thereof) of current feature steering methods
Cite this project
@misc{caimle2024clear,
title = {{Clear Thought and Clear Speech: Reducing Grammatical Scope Ambiguity}},
author = {Zmavli Caimle},
year = {2024},
month = nov,
note = {Submitted to Reprogramming AI Models Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/clear-thought-and-clear-speech-reducing-grammatical-scope-ambiguity}},
url = {https://apartresearch.com/sprints/projects/clear-thought-and-clear-speech-reducing-grammatical-scope-ambiguity}
}More from Reprogramming AI Models Hackathon
- 1st place by peer reviewView project: AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AutoSteer: Weight-Preserving Reinforcement Learning for Interpretable Model Control
AI Safety Initiative Groningen
Traditional fine-tuning methods for language models, while effective, often disrupt internal model features that could provide valuable insights into model behavior. We present a novel approach combining Reinforcement …
- View project: Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Classification on Latent Feature Activation for Detecting Adversarial Prompt Vulnerabilities
Feature Disruption Lab
We present a method leveraging Sparse Autoencoder (SAE)-derived feature activations to identify and mitigate adversarial prompt hijacking in large language models (LLMs). By training a logistic regression classifier on …
- View project: Utilitarian Decision-Making in Models - Evaluation and Steering
Utilitarian Decision-Making in Models - Evaluation and Steering
Byte To Brain
We design an eval based on the Oxford Utilitarianism Scale (OUS) that measures the model’s deontological vs utilitarian preference in a nine question, two factor model. Using this scale, we measure how feature steering …