Skip to content
Apart Research
Apart Research
Contact us
Sprint projectSep 17, 2024

nnsight transparent debugging

Aaron Kaufman, David Steinberg

Submitted to ARENA 4.0 Interpretability Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: nnsight transparent debugging

Open in a new tab (opens in new tab)
Share

We started this project with the intent of identifying a specific issue with nnsight debugging and submitting a pull request to fix it. We found a minimal test case where an IndexError within a nnsight run wasn’t correctly propagated to the user, making debugging difficult, and wrote up a proposal for some pull requests to fix it. However, after posting the proposal in the discord, we discovered this page in their GitHub (https://github.com/ndif-team/nnsight/blob/2f41eddb14bf3557e02b4322a759c90930250f51/NNsight_Walkthrough.ipynb#L801, ctrl-f “validate”) which addresses the problem. We replicated their solution here (https://colab.research.google.com/drive/1WZNeDQ2zXbP4i2bm7xgC0nhK_1h904RB?usp=sharing) and got a helpful stack trace for the error, including the error type and (several stack layers up) the line causing it.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Reviews of strong projects are shared with funders, as you choose below.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Decline to score

Only when the quality of the writing makes the work impossible to follow. Say why in your private critique; nothing shows on this page.

Sharing your name *

When we share a strong project with funders, your name, scores and public critique go with it, confidentially; this choice is only about whether your name is public. If you think this project deserves funding, please also endorse it: that carries the most weight.

Share my private critique with funders *

No public critique yet.

Cite this project

@misc{kaufman2024nnsight,
  title = {{nnsight transparent debugging}},
  author = {Aaron Kaufman and David Steinberg},
  year = {2024},
  month = sep,
  note = {Submitted to ARENA 4.0 Interpretability Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/nnsight-transparent-debugging}},
  url = {https://apartresearch.com/sprints/projects/nnsight-transparent-debugging}
}

More from ARENA 4.0 Interpretability Hackathon

All 4 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026