Skip to content
Apart Research
Apart Research
Contact us
Sprint projectJun 22, 2026Bengaluru

Empirical Verification of Topological Phase Transitions During Grokking

Sharvani Laxmi Somayaji · Team grokking_code

Submitted to Global South AI Safety Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Empirical Verification of Topological Phase Transitions During Grokking

Open in a new tab (opens in new tab)
Share

A multi-metric replication of the WeightWatcher framework analyzing the spectral properties and circuit complexity of neural networks during the grokking phase change. This project empirically isolates the exact mathematical inflection point where a network abandons dense memorization in favor of a sparse, generalizing circuit. Furthermore, it establishes a theoretical framework for future objective function ablation to test how loss landscape geometries dictate this topological collapse.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

How much would this matter for AI safety if it worked? How innovative is it? For scores of 4-5: is this actually new to the field, or replicating recent work?

Scoring guide
  1. 1Negligible. No clear problem addressed, or no meaningful novelty.
  2. 2Limited. Addresses a real problem but with a generic or well-trodden approach. Incremental at best.
  3. 3Moderate. Clear problem with a reasonable approach; some novelty in framing or method beyond routine application of existing tools.
  4. 4Significant. Important problem with an original approach, or identifies a neglected problem area. A valuable contribution others could build on.
  5. 5Exceptional. Tackles a critical AI safety problem with a genuinely novel approach, or opens a new research direction. Clear theory of change. You'd be excited to share this with researchers in the area.

How sound are methodology, implementation, and findings?

Scoring guide
  1. 1Seriously flawed. Methodology broken, results uninterpretable, or implementation doesn't work.
  2. 2Weak. Approach has significant gaps: missing validation, flawed experimental design, or incomplete implementation.
  3. 3Competent. Technically solid given the short duration. Methodology makes sense, results are interpretable, limitations acknowledged, work builds toward clear conclusions.
  4. 4Strong. Thorough methodology with convincing validation. Results clearly support conclusions. Immediately useful for future work.
  5. 5Exceptional. Ambitious scope executed rigorously. Surprising findings, novel methods, or unusually robust validation.

How clearly are work, findings, and impact potential communicated?

Scoring guide
  1. 1Incomprehensible. Cannot determine what the project is actually claiming or doing.
  2. 2Hard to follow. Key information buried, missing, or diluted by excessive length. Significant effort to extract main points.
  3. 3Clear enough. Can understand the problem, approach, and results without undue effort. Core content clearly present: problem, method, findings, limitations.
  4. 4Well presented. Easy to follow, well-structured, appropriate level of detail. Target audience would get it quickly.
  5. 5Exceptionally clear. A pleasure to read. Complex ideas made accessible. Could serve as a model for how to present this type of work.

  1. This paper proposes to study grokking through the lens of spectral analysis and heavy tailed weight matrix properties, framing the memorization to generalization transition as a topological phase change. The idea of connecting WeightWatcher's power law exponents to the grokking phenomenon is reasonable and could yield insight if executed fully.

    The fundamental problem is that the paper is incomplete. Section 3, which contains the actual proposed contributions (three hypotheses about how objective function changes affect the phase transition), is entirely unvalidated. The authors acknowledge this in Section 4, but the result is a submission that is one third replication of existing tools (WeightWatcher on MNIST) and two thirds speculation. For a hackathon submission, running at least one ablation (removing weight decay is trivial) would have substantially changed the paper's standing.

    The baseline replication itself raises questions. The claim of a "distinct phase transition" on MNIST with a standard MLP is unusual. Grokking is typically demonstrated on algorithmic tasks (modular arithmetic, permutation groups) where the gap between memorization and generalization is stark and delayed. On MNIST with a width 200 network, the train/test accuracy gap closes quickly under normal training. The paper claims training accuracy hits 1.0 by step 10^3 while test accuracy continues climbing slowly, but does not show whether this gap is large enough to constitute grokking rather than ordinary generalization dynamics. No figures or raw numbers are provided, only qualitative descriptions of trends.

    The theoretical framing uses language ("topological collapse," "memorization basin," "algorithmic circuit") that sounds precise but is not backed by formal definitions or measurements that would distinguish these claims from standard observations about regularization and weight decay. Greedy Circuit Complexity dropping to near zero is stated without explaining what this metric actually computes or why its collapse should be interpreted as circuit formation rather than, say, rank collapse.

    The writing is confident and structured but overreaches relative to what was actually done.

    Read full reviewShow less

  2. This project explores an interesting mechanistic interpretability question by examining whether grokking corresponds to measurable topological changes in neural network weight matrices using spectral analysis. The emphasis on validating spectral signatures before proposing interventions is methodologically sound, and connecting WeightWatcher metrics to grokking is a worthwhile direction.

    As next steps, the project could:

    - Implement the proposed objective-function ablations, as these constitute the primary novel contribution and would substantially strengthen the paper.

    - Evaluate multiple architectures, datasets, and random seeds to demonstrate that the observed spectral signatures are robust rather than specific to one training configuration.

  3. This small paper shows that a delayed generalization in a small MNIST MLP is accompanied by a shift in metrics. I would have wanted to see the WW alpha compared to HTSR baselines so that we can start comparing this hypothesis with alternative theories.

    The larger point of movement from memorization to more generalization will need a diverse dataset beyond just MNIST

    I would be interested to see confidence intervals, and clear theory of impact for this research for AI Safety beyond what is already known in this domain

Cite this project

@misc{somayaji2026empirical,
  title = {{Empirical Verification of Topological Phase Transitions During Grokking}},
  author = {Sharvani Laxmi Somayaji},
  year = {2026},
  month = jun,
  note = {Submitted to Global South AI Safety Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/empirical-verification-of-topological-phase-transitions-during-grokking-nmp5}},
  url = {https://apartresearch.com/sprints/projects/empirical-verification-of-topological-phase-transitions-during-grokking-nmp5}
}

More from Global South AI Safety Hackathon

All 216 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026