Quantization-Conditioned Alignment Degradation
Krishna Venkatesh
Post-training quantization enables language model deployment on edge hardware across the Global South, yet its effect on safety alignment remains unstudied. We benchmark Attack Success Rate (ASR) on Llama 3.1 8B across four GGUF quantization levels (Q8, Q5, Q4, Q3) against a full BF16 precision control group served via Groq, using 150 prompts from the AdvBench Harmful Behaviors dataset and an LLM-as-judge scoring protocol. We find that ASR remains flat across all GGUF quantization levels (0.7% for
Q3 through Q8), while full-precision BF16 models served via API exhibit substantially higher ASR (7.3% for Llama 3.1 8B, 11.3% for Llama 3.3 70B), indicating that serving infrastructure, not quantization precision but drives the primary variation in refusal behaviour. These results directly inform deployment standards for resource-constrained settings where Q4 and Q3 quantization are practical necessities. Our fully reproducible pipeline is open-source and extensible to other model families.
The biggest problem is simple, you say anyone can reproduce your work, but you didn't actually include your results, so nobody can check a single number, and I'd start by just posting them. You should also clear up a confusing mix-up about which AI did the grading, and make sure the grader isn't the same AI you're testing, since that's a bit like having a student mark their own exam. Your most valuable finding is almost an accident, which is that the same AI behaves differently depending on the software you run it through, not on how much you shrink it, so I'd test that head-on by running the exact same AI two ways and changing nothing else. And honestly, your test is too easy, because the AIs refused almost everything, so there's no real difference to see, which means you should try harder, trickier attacks before you can safely conclude that shrinking the AI is harmless.
No feedback
Cite this work
@misc {
title={
(HckPrj) Quantization-Conditioned Alignment Degradation
},
author={
Krishna Venkatesh
},
date={
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}


