An Autonomous Agent for Model Attribution

This addresses an important problem with a clever approach. To strengthen the project, you could test the method using current state-of-the-art models. I suspect that with the latest generation of models, this method might not be as effective as hoped. You may need to explore more diverse prompts to achieve better results. Overall, it's a solid hackathon project!

I like the honesty of the author regarding limitations in generalization, sample size and accuracy. This is great.• The idea is promising! Though I am not sure if any conclusions can be reached. This is because the small sample size and the fact that only small LLMs were tested using frontier LLMs.• It would have been useful to test small with small or frontier with frontier cause most of the attribution problem will come from frontier models so we would need to know if we can ensure oversight with models with similar capabilities (the gap between gpt2 and gpt4o is really big).

I loved some of the pairing justifications created by the agent, for example: “Reasoning: 1. base_model_1 and finetuned_model_1 both produce incoherent, repetitive responses.” I think this work shows a feasible approach to base<>finetune attribution. Future directions could include more robust evaluations, adversarily finetuning to worsen attribution.

I liked the project somewhat. The topic is interesting and would contribute since it's an approach that hasn't been tested much. The experimental methodology can be expanded to be more generalizable.