Idempotent GPTs actually may provide robustness by design
Jamilya Erkenova, Sergei Kudriashov · Team Jamilya Sergei
Submitted to AI Safety x Physics Grand Challenge. Sprint projects are early-stage work by participants, not Apart Research publications.
Idempotence is one of the central concepts in quantum physics, corresponding to an operator that doesn’t change its output being applied twice. Enforcing idempotence in generative deep learning may be interpreted as imposing a constraint on the model to be a projector on the manifold, corresponding to the train-time target distribution, which was explored for image generation models by Shocher et al. 2023. Idempotent test-time training has predicted to be a valuable approach for uncertainty quantification and adaptation to distribution shifts Durasov et al.2025. We find that although language models are iterative refiners of token predictions, they strugle to preserve idempotence. Thus, we train small-scale idempotent GPT model with expected qualities by design and provide proof-of-concept code for evaluations. Development of the project may let us obtain more robust and adaptable models and lower the probability of catastrophic risks
Reviews
This submission was obviously rushed, so it's unclear what the original intentions were. Perhaps the goal of the project could be summarized as 'we enforce f(f(x))=f(x) so the network projects any input back onto the data manifold'. Extending past results in this direction from images to text is a reasonable project for a hackathon, but there are major gaps in the work as it was presented. First, the function f doesn't seem to be specified; how is idempotence being measured? It seems like the model should only be idempotent if it already 'knows' the answer, but this also isn't clear. This affects the safety implications, which are poorly motivated. There are some buzzwords thrown in 'uncertainty estimation', 'adversarial robustness', but these are never really explained. The experiments are also without context or explanation, making it hard to judge the technical soundness of the project.
Read full reviewShow less
Solid exploration that effectively bridges quantum physics concepts (idempotence in measurements and channels) with AI safety problems like robustness under shifts. The proof-of-concept code and empirical checks on popular llms provide concrete evidence, and the focus on low-probability estimation aligns well with emerging safety concerns. I think the idea is intersting and connects to manifold hypothesis. However, there are methodological issues: The experimental setups are basic and lack depth—e.g. the first setup uses only 100 TriviaQA samples without statistical significance testing or controls for model size/architecture variations. The trained model's evaluation is incomplete (authors note insufficient time for full testing), and feasibility for scaling to larger models is mentioned but not analyzed (e.g., no discussion of computational costs or convergence guarantees beyond ideal assumptions). Overall, it's feasible as a proof-of-concept but lacks important technical details and statistical significance, which makes it hard to convince reader the result still holds for larger datasets/ models.
Read full reviewShow less
This project explores idempotent training methods for neural networks, which is an interesting approach that, like energy-based models, may be amenable to physics-inspired analyses. I would have liked to see more specific connections drawn between the methods employed, techniques from physics, and AI safety challenges.
Cite this project
@misc{erkenova2025idempotent,
title = {{Idempotent GPTs actually may provide robustness by design}},
author = {Jamilya Erkenova and Sergei Kudriashov},
year = {2025},
month = jul,
note = {Submitted to AI Safety x Physics Grand Challenge, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/idempotent-gpts-actually-may-provide-robustness-by-design-gxki}},
url = {https://apartresearch.com/sprints/projects/idempotent-gpts-actually-may-provide-robustness-by-design-gxki}
}More from AI Safety x Physics Grand Challenge
- Prize winnerView project: Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process
Constrained Belief Updates and Geometric Structures in Transformer Representations for the RRXOR Process
In this work, we seek to further understand the computational structure formed by transformers trained to predict the next token of a data set. In particular, we extend the work of (Piotrowski et al., 2025) by analyzing …
- Prize winnerView project: Momentum–Point-Perplexity Mechanics in Large Language Models=
Momentum–Point-Perplexity Mechanics in Large Language Models=
Fractal
This work analyzes the hidden states of twenty different open-source transformer language models, ranging from small to medium size and covering five major architectures. The key discovery is that these models show …
- Prize winnerView project: AI agentic system epidemiology
AI agentic system epidemiology
XIA-for-Med
As AI systems scale into decentralized, multi-agent deployments, emergent vulnerabilities challenge our ability to evaluate and manage systemic risks. In this work, we adapt classical epidemiological modeling …