Evaluating Safety Judge Design Against Adversarial Attacks

Thank you for your submission. Code looks well structured and documented. Sadly, with an incomplete write-up I can’t see what you achieved.

This may be applicable: At the start of a time-boxed task, it is important to plan out what the task phase are, when you need to start/end each task phase so you don’t run out of time.

While you didn’t get time to finish this , I’d encourage you that it’s an interesting line of work to evaluate adversarial robustness of the judges to prompts. I don’t think the unintended side effects should worry you too much - rather, I think it’s an interesting side feature in its own right.

There are a number of prompt optimization / modification techniques that you can explore to see how effective they are, particularly at dspy.(https://github.com/stanfordnlp/dspy)

Constructive critique:

Strength:

Good proposal: adversarially manipulate the judge and see which features of the judge determine the score and use these interpretable properties to further improve the judge.

Weakness:

There does not seem to be much work to show progress on the project (judge model is not identified, the same applies to judge construction details and mech interp experiments design)

Limited implementation

Expert Orchestration: 1

MI: 1

Technical Imp. and reproducibility: 1

I like that this project increases the interpretability of the safety assessment criteria. However, I predict that this method might get more difficult as models become more sophisticated.

Unfortunately, the report appears incomplete and there's a lot I would have liked to see--including evaluations of how useful and reliable the keyword assessment method is.