Feb 2, 2026
DOMAIN OWNERSHIP PROBING
Rijal Saepuloh, Ifeoma Ilechukwu, Valmik nahata, Saahir Vazirani
We propose Domain Ownership Probing (DOP), a lightweight verification method that evaluates a model’s internal representation structure instead of its stochastic text outputs. DomainProbe embeds domain-specific statements, forms prototype centroids, and computes domain ownership win-rate and cohesion to assess whether knowledge domains are consistently encoded. By adding an auto-tuned layer search, the method remains effective for both encoder models and decoder-only LLMs, supporting practical AI governance and compliance verification without exposing training datasets.
The team clearly shows technical aptitude — the pipeline is well-constructed and the implementation appears sound. The core problem is that what this method demonstrates is that embeddings of topically similar statements are close together in embedding space, which is a well-established property of language model representations. It's not clear that this gives us a way to distinguish between safe models and unsafe models — only between good models and bad models. Any competent model will cluster biology statements together; one that doesn't is simply poorly trained. My core recommendation would be to think about what kind of probes would yield actionable information for auditors, and work backward from there.
Thank you for pointing to embedding geometry as a stable and privacy-preserving signal for capability verification. I am excited about work that enables model assessment without exposing proprietary data or relying on gameable benchmarks. The main gap is in connecting this geometric signal to dangerous capabilities evaluation. A model clustering biology probes together tells us it has structured representations of biology, but it does not tell us whether the model can provide bioweapons development uplift.
Cite this work
@misc {
title={
(HckPrj) DOMAIN OWNERSHIP PROBING
},
author={
Rijal Saepuloh, Ifeoma Ilechukwu, Valmik nahata, Saahir Vazirani
},
date={
2/2/26
},
organization={Apart Research},
note={Research submission to the research sprint hosted by Apart.},
howpublished={https://apartresearch.com}
}
