DOMAIN OWNERSHIP PROBING

Thank you for pointing to embedding geometry as a stable and privacy-preserving signal for capability verification. I am excited about work that enables model assessment without exposing proprietary data or relying on gameable benchmarks. The main gap is in connecting this geometric signal to dangerous capabilities evaluation. A model clustering biology probes together tells us it has structured representations of biology, but it does not tell us whether the model can provide bioweapons development uplift.

The team clearly shows technical aptitude — the pipeline is well-constructed and the implementation appears sound. The core problem is that what this method demonstrates is that embeddings of topically similar statements are close together in embedding space, which is a well-established property of language model representations. It's not clear that this gives us a way to distinguish between safe models and unsafe models — only between good models and bad models. Any competent model will cluster biology statements together; one that doesn't is simply poorly trained. My core recommendation would be to think about what kind of probes would yield actionable information for auditors, and work backward from there.