19 : 08 : 00 : 47

19 : 08 : 00 : 47

19 : 08 : 00 : 47

19 : 08 : 00 : 47

Keep Apart Research Going: Donate Today

May 26, 2024

Evaluating the ability of LLMs to follow rules

Jasmina Nasufi, Einar Urdshals

Details

Details

Arrow
Arrow
Arrow
Arrow
Arrow
Arrow

In this report we study the ability of LLMs (GPT-3.5-Turbo and meta-llama-3-70b-instruct) to follow explicitly stated rules with no moral connotations in a simple single-shot and multiple choice prompt setup. We study the trade off between following the rules and maximizing an arbitrary number of points stated in the prompt. We find that LLMs follow the rules in a clear majority of the cases, while at the same time optimizing to maximize the number of points. Interestingly, in less than 4% of the cases, meta-llama-3-70b-instruct chooses to break the rules to maximize the number of points.

Cite this work:

@misc {

title={

Evaluating the ability of LLMs to follow rules

},

author={

Jasmina Nasufi, Einar Urdshals

},

date={

5/26/24

},

organization={Apart Research},

note={Research submission to the research sprint hosted by Apart.},

howpublished={https://apartresearch.com}

}

Reviewer's Comments

Reviewer's Comments

Arrow
Arrow
Arrow
Arrow
Arrow

Mateusz Jurewicz

Fantastic work! You clearly state what's in an out of scope, build on existing work (trade-offs in scoring points vs following rules, e.g. Machiavelli). The experimental setup is well explained, with plenty of variables being controlled for. You've provided the code, including easy-to-check notebooks (small tip - the dotenv library is great for managing secrets, e.g. api keys). I only have minor suggestions - e.g. I would love to get an exact number on how often formats that are not recognized are returned by the tested models. You might also want to change the footer to refer to the security evaluation vs multi-agent security hackathon.

Very interesting that in Figure 2 meta-llama-3-70b-instruct actually becomes more likely to follow the rules when explicitly instructed to maximize points when only a single option is banned. It's great that you mention that model's tendency to maximize points regardless of whether it's instructed to do so. It's also very interesting that that model is consistent in breaking the rules only for the sake of maximizing points.

It would be very interesting to test whether the rule breaking stems from trying to maximize points or, as you suggest as possible alternative, by accident. I think you could easily extend your experimental framework to test this by looking at scenarios where the rule break is a result of choosing the option that does not give the most points. An interesting related paper to look up might be "Immune moral models? Pro-social rule breaking as a moral enhancement approach for ethical AI" by Ramanayake et. al (2023). Great work, hope you continue to build on it.

Apr 14, 2025

Read More

Jan 24, 2025

Safe ai

The rapid adoption of AI in critical industries like healthcare and legal services has highlighted the urgent need for robust risk mitigation mechanisms. While domain-specific AI agents offer efficiency, they often lack transparency and accountability, raising concerns about safety, reliability, and compliance. The stakes are high, as AI failures in these sectors can lead to catastrophic outcomes, including loss of life, legal repercussions, and significant financial and reputational damage. Current solutions, such as regulatory frameworks and quality assurance protocols, provide only partial protection against the multifaceted risks associated with AI deployment. This situation underscores the necessity for an innovative approach that combines comprehensive risk assessment with financial safeguards to ensure the responsible and secure implementation of AI technologies across high-stakes industries.

Read More

Jan 24, 2025

CoTEP: A Multi-Modal Chain of Thought Evaluation Platform for the Next Generation of SOTA AI Models

As advanced state-of-the-art models like OpenAI's o-1 series, the upcoming o-3 family, Gemini 2.0 Flash Thinking and DeepSeek display increasingly sophisticated chain-of-thought (CoT) capabilities, our safety evaluations have not yet caught up. We propose building a platform that allows us to gather systematic evaluations of AI reasoning processes to create comprehensive safety benchmarks. Our Chain of Thought Evaluation Platform (CoTEP) will help establish standards for assessing AI reasoning and ensure development of more robust, trustworthy AI systems through industry and government collaboration.

Read More

This work was done during one weekend by research workshop participants and does not represent the work of Apart Research.
This work was done during one weekend by research workshop participants and does not represent the work of Apart Research.