An Interpretable Classifier based on Large scale Social Network Analysis

Mar 10, 2025

An Interpretable Classifier based on Large scale Social Network Analysis

Monojit Banerjee

Read Project

Details

Summary

Mechanistic model interpretability is essential to understand AI decision making, ensuring safety, aligning with human values, improving model reliability and facilitating research. By revealing internal processes, it promotes transparency, mitigates risks, and fosters trust, ultimately leading to more effective and ethical AI systems in critical areas. In this study, we have explored social network data from BlueSky and built an easy-to-train, interpretable, simple classifier using Sparse Autoencoders features. We have used these posts to build a financial classifier that is easy to understand. Finally, we have visually explained important characteristics.

Download PDF

View Presentation

Cite this work:

@misc {

title={

An Interpretable Classifier based on Large scale Social Network Analysis

author={

Monojit Banerjee

date={

3/10/25

organization={Apart Research},

note={Research submission to the research sprint hosted by Apart.},

howpublished={https://apartresearch.com}

}

Review

Reviewer Name

Constructive critique

Criteria 1:

Advancing interpretability (Possible questions to answer in this criteria: Does the project contribute to the field of mechanistic interpretability? Does it provide new insights into understanding or steering AI model behavior? How well does it move us towards reprogramming AI models? How original and innovative is the approach?)

2.5

Criteria 2:

Relevance for AI safety (Possible questions to answer in this criteria: How important is the contribution to advancing the field of AI safety? Do we expect the results to generalize beyond the specific case(s) presented in the submission? Does the approach introduce new safety mechanisms or enhance existing ones in innovative ways?))

2.5

Criteria 3:

Methodology & presentation (Possible questions to answer in this criteria: How well is the project executed from a technical standpoint?, Is the code well-structured, documented, and reproducible?, How effectively does it utilize Goodfire's SDK/API and other provided resources?, How clearly and effectively is the research presented in the paper?, Quality of visualizations and demos (if applicable), Clarity of methodology explanation and results interpretation))

2.5

Reviewer's Comments

Natalia Perez-Campanero

The project addresses an important problem by trying to make financial sentiment analysis more interpretable. The use of SAEs and decision trees is a reasonable approach for achieving interpretability. The paper is clearly written and structured, but could do with more details on implementation and the threat model, which is currently somewhat vague - a more detailed discussion of potential failure modes and mitigation strategies would strengthen the AI safety aspect. It would also be valuable to include a more thorough error analysis and discussion of the limitations of the approach. I'd encourage you to dive a bit more into the literature on mech interp to help with the analysis of the SAE features, and/or adversarial robustness in NLP to inform strategies for making similar systems more resilient to manipulation.

C1:

C2:

2.5

C3:

Zainab Majid

Good work over the hackathon - well done, and interesting approach using a novel social media platform! I've seen references to current research re. SAEs and decision tree classifiers, I'd recommend engaging with the available literature more thoroughly - potentially with a focus on finance. It's definitely interesting to think about the interpretability of sentiment analysis but moving forwards, I'd spend some time detailing all potential applications and impacts to the AI safety space. As is stated in the 'Discussion and Conclusion' section, future work should use data outside of just BlueSky / social media to broaden the research - I think this should be a priority. It would also be interesting to compare these results to more traditional interpretability techniques.

C1:

C2:

C3:

Recent Projects

View All

Mar 24, 2025

Attention Pattern Based Information Flow Visualization Tool

Understanding information flow in transformer-based language models is crucial for mechanistic interpretability. We introduce a visualization tool that extracts and represents attention patterns across model components, revealing how tokens influence each other during processing. Our tool automatically identifies and color-codes functional attention head types based on established taxonomies from recent research on indirect object identification (Wang et al., 2022), factual recall (Chughtai et al., 2024), and factual association retrieval (Geva et al., 2023). This interactive approach enables researchers to trace information propagation through transformer architectures, providing deeper insights into how these models implement reasoning and knowledge retrieval capabilities.

Mar 24, 2025

jaime project Title

bbb

Mar 25, 2025

Safe ai

The rapid adoption of AI in critical industries like healthcare and legal services has highlighted the urgent need for robust risk mitigation mechanisms. While domain-specific AI agents offer efficiency, they often lack transparency and accountability, raising concerns about safety, reliability, and compliance. The stakes are high, as AI failures in these sectors can lead to catastrophic outcomes, including loss of life, legal repercussions, and significant financial and reputational damage. Current solutions, such as regulatory frameworks and quality assurance protocols, provide only partial protection against the multifaceted risks associated with AI deployment. This situation underscores the necessity for an innovative approach that combines comprehensive risk assessment with financial safeguards to ensure the responsible and secure implementation of AI technologies across high-stakes industries.

Mar 24, 2025