Reprogramming AI Models Hackathon

Use this template for your submission [Required]

‍

Submission Requirements

Each team should submit a research paper that includes:

1. Project title and team members
2. Executive summary (max 250 words)
3. Introduction and problem statement
4. Methodology and approach
5. Results and analysis
6. Discussion of implications for AI interpretability
7. Conclusion and future work
8. References

Additionally, teams should provide:

• A link to their code repository (e.g., GitHub)
• Any demo materials or visualizations (if applicable)

Evaluation Criteria

Submissions will be judged based on the following criteria:

1. Interpretability Advancement
   • Does the project contribute to the field of AI interpretability?
   • Does it provide new insights into understanding or steering AI model behavior?
   • How well does it align with the hackathon's focus on reprogramming AI models?
2. Research Quality
   • How original and innovative is the approach?
   • Does it present novel ideas or combine existing techniques in unique ways?
   • Do we expect the results to generalize beyond the specific case(s) presented in the submission?
3. Technical Implementation
   • How well is the project executed from a technical standpoint?
   • Is the code well-structured, documented, and reproducible?
   • How effectively does it utilize Goodfire's SDK/API and other provided resources?
4. Presentation and Communication
   • How clearly and effectively is the research presented in the paper?
   • Quality of visualizations and demos (if applicable)
   • Clarity of methodology explanation and results interpretation

Key Points to Remember:

• Focus on creating tools that provide meaningful insights into model behavior or enable precise modifications.
• Consider how your approach might scale to larger, more complex AI systems.
• Ethical considerations should be at the forefront of your design process.

💡 Frequently Asked Questions

General Questions

Q: What is the timeline for the hackathon?

A: The hackathon runs from Friday, 22nd November, through Monday, 25th November(3 AM UTC). There will be two brainstorming sessions (Tuesday and Thursday), with API access beig granted starting Thursday morning.

Q: What team size is recommended?

A: Teams should be 2-4 people, with a maximum of 5. While individual submissions are allowed, team participation is encouraged for better project outcomes.

Q: How do we get access to the API?

A:

1. Form your team
2. Have the team lead fill out the registration form
3. Receive API access credentials (starting Thursday morning 21st November)

Technical Questions

Q: Which models are available through the SDK?

A: The SDK provides access to:

• Llama 3 8B
• Llama 3 70B

Q: What are the main capabilities of the Goodfire SDK?

A: The SDK provides:

• SAE feature inspection
• Feature interventions
• Feature search functionality
• Contrast analysis between datasets
• Feature activation analysis

Q: Are there rate limits for the API?

A: Yes, there are rate limits, but they can be increased upon request with justification. The SDK implements exponential backoff to help manage these limits.

Q: Can we access raw neuron activations?

A: No, the SDK only provides access to feature-level information, not raw neuron activations.

Q: Can we combine Goodfire's SDK with other tools?

A: Yes, you're encouraged to use any additional tools alongside the SDK to achieve your project goals.

Feature-Related Questions

Q: How can we find relevant features for our task?

A: The SDK provides several methods:

1. Search: Query features using string searches
2. Inspect: Analyze feature activations for specific inputs
3. Contrast: Compare features between different datasets

Q: Can we modify feature activations?

A: Yes, you can perform feature interventions to modify model behavior, including conditional interventions.

Q: Are the SAE features proprietary?

A: Yes, Goodfire's SAE features are proprietary but may be open-sourced in the future.

Support & Resources

Q: Where can we get help during the hackathon?

A: You can:

1. Use the help desk channel on Discord
2. Contact the technical support team directly
3. Attend office hours (Saturday/Sunday)
4. Participate in hack talks

Q: What if we need increased rate limits?

A: Submit a request through the provided form with justification, and the team will review it case by case.

Q: Are there example notebooks available?

A: Yes, Goodfire provides several example notebooks, including:

• Feature exploration examples
• Decision tree with features
• Latent space exploration

Q: How can we submit feature requests or feedback?

A: While immediate changes during the hackathon may not be possible, Goodfire welcomes all feedback and feature requests for future improvements.
‍