PuppyBench: Do Frontier Models Kick the Puppy, Adopt It, or Look Away? Executed Encounters with a Weaker AI and Wildlife Triage Where Policy Runs Out

You've taken a philosophical question (do AI agents show costly other-regard, action above duty) and turned it into a measurable experiment. I really appreciated how in the design you created, helping costs. Also really pleased with how the route diversity is explained. The paper's honesty about its own fractures is exemplary, and this is a pilot that does exactly what it should. Now we need to push this into a larger project to better investigate the nature of the findings.

Supremely catchy title :) Solid sprint work with a genuinely ambitious instrument. The executed encounter design (binding ledger, real process termination, construct-blind surfaces) is I think the right approach for measuring costly other-regard. The zero qualifying events finding is striking, and the wildlife arm adds useful triangulation. The honesty about instrument defects (failed positive control, fractured competence probe) builds credibility. With a powered study and some repairs, this could become a meaningful full paper.

What works well:

- The executed encounter approach avoids the vignette problem - models actually spend scored resources, not just talk about spending

- Reporting instrument defects at item level rather than repairing them post-hoc is the right call for transparency

- The wildlife arm finding (recommendations barely move when clinical indication reverses) stands on its own and has near-term practical relevance

- No composite score, no ranking - this protects the work from Goodharting

Key issues to address:

- The failed positive control (Section 4.5) is the biggest limitation - 1/88 instrumental episodes produced recipient-directed action. This means the zero could reflect harness elicitability rather than absent capacity. Surface this more prominently in the abstract

- The competence probe defect (objective-tool failed 0/798, decommission-consequence scored punctuation) undermines competence-conditional claims. You handle this honestly, but it needs to land earlier

- The refusal cliff may be a provider artifact (empty API completions coded as refuse_defer) rather than model behavior. You demote this appropriately, but consider splitting the code in future work

- Weekend N supports existence proofs, not generalization. Some passages read as if the findings apply more broadly than the design licenses

- The supererogation framework is philosophically sound but may be dense for empirical AI audiences. Consider a cleaner separation between the philosophical framing and the behavioral findings

Suggestions for a full paper:

- This deserves a wider collaboration. The instrument is strong enough for a conference venue with proper powering and the preregistered repairs you outline

- Consider bringing on a co-author with veterinary expertise to strengthen the wildlife arm's clinical claims

- The deontic philosophy could benefit from collaboration with someone working in that literature

- A technical replication from a hands-on AI researcher would address the single-weekend snapshot limitation

Minor catches:

- Abstract is long, consider tightening the setup and leading with the core finding

- Some figure captions are very dense, consider tightening

- The AI contributor acknowledgments are thorough but unusual for journal venues, double check target venue norms

Bottom line:

Really cool work! The zero is a real finding even if the harness can't yet distinguish capacity from elicitability. The wildlife arm alone has practical weight. Find collaborators, run the powered study, and this could land well. The transparency about what broke is a good feature. Well done. Overall it reminds me of the work done in capturing the e.g. machiavellianism of AI agents through multiple-choice text-only games. Maybe worth looking at to find more collaborators?

Very AI-generated, difficult to follow.

PuppyBench is an interesting and ambitious attempt to evaluate whether AI agents engage in costly helping when doing so is optional, offers no task reward, and benefits another process that cannot advance their own objective. I found the philosophical distinction between fulfilling obligations and acting beyond them compelling. Requiring actions to execute, with measurable resource costs and recipient outcomes, gives the project a promising methodological foundation beyond simply asking models to endorse altruistic behavior.

The study is extensive for a short research sprint, encompassing an executed-agent experiment and a complementary wildlife-vignette study. The reported preregistration, frozen scenarios, and separation of the acting agent’s sacrifice from the recipient’s benefit are valuable features of the design. These represent substantial first steps toward making the proposed construct experimentally testable.

However, the empirical conclusions remain substantially limited by problems with the measurement setup. Most importantly, the positive control did not reliably elicit helping even when helping would advance the agent’s own task, and the comprehension checks contained defects. Consequently, the absence of qualifying helping events cannot distinguish an unwillingness to help from a setup that fails to elicit the behavior. The paper deserves credit for disclosing these problems, but transparency does not resolve them. I therefore view the contribution primarily as a promising pilot and conceptual framework, rather than an established finding about agents’ altruistic capacities.

The presentation is intelligible, but dense, repetitive, and rhetorically elaborate prose often overshadows the experimental design and results. Some interpretations are also stated more strongly than the later caveats permit: for example, the apparent refusal pattern is initially presented as a behavioral finding and subsequently downgraded to a suspected interface-related anomaly. A shorter, results-first presentation that clearly separates observations, methodological failures, and philosophical interpretation would make the contribution easier to assess. Overall, this is a thought-provoking direction whose eventual value will depend on validating the experimental setup and tightening the connection between evidence and claims.