Skip to content
Apart Research
Apart Research
Contact us
Sprint projectNov 24, 2025Cambridge

Opening Doors to Multimodal Deception

Takeshi Miki, Don Yin · Team Happy VLM

Submitted to Defensive Acceleration Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.

Read the report

Report: Opening Doors to Multimodal Deception

Open in a new tab (opens in new tab)
Code (opens in new tab)
Share

Organizations increasingly deploy vision-language models (VLMs) locally to protect sensitive data, yet there is almost no empirical work on whether these multimodal systems can systematically lie about visual content or whether text-based safety tools transfer to visual contexts. We present, to our knowledge, the first systematic study of deception detection in VLMs. First, we construct a VLM deception benchmark with 1,048 image–text pairs derived from COCO and show that a standard VLM can be reliably induced to contradict ground-truth visual information, while a linear probe on hidden activations detects these multimodal lies with 98.7% AUROC. Second, we perform a geometric analysis of deception representations across a text-only LLM and a VLM, finding high alignment (cosine similarity > 0.85) between deception directions in the text backbone and in the VLM text stream, suggesting that deception is encoded as a modality-independent concept rather than a purely task-specific feature. Third, we demonstrate cross-modal transfer for deception detection: a probe trained only on text-based deception data attains 78.5% AUROC on visual deception, corresponding to 57% transfer efficiency. These results indicate that text-trained probes can provide useful zero-shot monitoring for local VLM deployments, while also highlighting open questions about modality-specific failure modes and the limits of cross-modal safety transfer.

Reviews

Judging this Sprint?

Review this project

Your public critique appears on this page without your name. Your private critique is not published; only the Apart team reads it. If you agree below, we share your review with grantmaking.ai (opens in new tab) and the Transformative AI Fund so strong projects can be funded.

Not shown on this page.

Shown on this page, without your name.

Only the Apart team reads this, and funders if you agree below.

Share my name publicly on grantmaking.ai *
Share my private critique with funders *

Does this reduce AI-related catastrophic or existential risks?

Scoring guide
  1. 1Minimal Impact. The project has minimal relevance to AI safety. It doesn't meaningfully address risks uniquely enabled or accelerated by advanced AI systems.
  2. 2Tangential Connection. The project touches on AI safety concepts but lacks depth or specificity. The connection to AI-enabled threats (bio, cyber, or AI misuse) is weak or unclear.
  3. 3Clear AI Safety Value. The project clearly reduces AI-related risks with valuable contributions. It addresses specific threats from AI systems and engages meaningfully with biosecurity, cybersecurity, or AI safety challenges.
  4. 4Significant Impact Potential. The above, plus the project demonstrates scalable safety mechanisms or defensive approaches. It shows clear potential to buy time for solving harder problems like alignment, or creates positive externalities for the broader AI safety ecosystem.
  5. 5Major Advancement. The above, plus the project represents a significant leap forward in defensive AI safety. Judges would eagerly share this with biosecurity, cybersecurity, or AI safety researchers and expect it to influence the field.

Does this strengthen the shield against AI-enabled threats?

Scoring guide
  1. 1Minimal Relevance. The project is only tangentially related to defensive technology or societal protection. Connection to biosecurity, cybersecurity, or defensive infrastructure is unclear or missing.
  2. 2Some Relevance. The project has some relevance to defensive acceleration, but the connection is broad or generic. It touches on defense without specific focus on AI-enabled threats or protective capabilities.
  3. 3Clear Relevance. The project clearly addresses defensive gaps against AI-enabled threats. It connects to at least one track (biosecurity, cybersecurity, or defense infrastructure) and demonstrates understanding of the threat landscape.
  4. 4Strong Contribution. The above, plus the project builds on existing defensive approaches and offers novel tools, frameworks, or implementations. It explicitly explains how it strengthens defensive capabilities with realistic deployment potential.
  5. 5Breakthrough Impact. The above, plus the project provides breakthrough insights or tools that could significantly influence defensive technology development. It identifies critical gaps and presents compelling solutions with clear paths from prototype to deployed system.

Did you build something that actually works?

Scoring guide
  1. 1Incomplete or Flawed. The project appears rushed or incomplete. Technical implementation is flawed, core functionality doesn't work, or the approach is fundamentally unsound. Little to no documentation.
  2. 2Basic Competence. The project shows reasonable effort with basic technical competence. Core functionality partially works. Documentation exists but may be incomplete. Some limitations are acknowledged.
  3. 3Solid Hackathon Project. The project is technically solid and well-scoped for 48 hours. Core functionality works and is documented. Code/methods are understandable and limitations are honestly addressed. This is what a good weekend prototype should look like.
  4. 4Impressive Implementation. The above, plus the implementation exceeds typical hackathon quality. Clear methodology, thorough documentation, and working demo. The tool/prototype is immediately useful for defenders and could realistically be built upon.
  5. 5Exceptional Execution. The project far exceeds expectations with exceptional technical execution. The implementation is elegant, fully functional, and includes something special (e.g., deployed demo, exceptional documentation, innovative architecture, or clear startup potential).

  1. Strengths: Most deception detection work has focused on text; extending to VLMs is timely as organizations deploy multimodal systems locally. The cross-modal transfer result is particularly valuable because it suggests existing AIS infra isn't starting from zero for multimodal monitoring. Was also very impressed with the benchmark construction and quantitative rigor relative to hackathon scope/timing.

    Suggestions: The defensive framing could be sharper. Why is detecting VLM deception harder than text deception? What's the attacker model—adversarial fine-tuning, prompt injection, something else? Explicitly connecting to the offensive/defensive asymmetry would strengthen the def/acc positioning. We'd also be curious about failure modes: when does the cross-modal transfer break down, and what does that tell us about where new defensive tools are needed?

    From Halcyon Ventures POV: I could see this integrated into model evaluation pipelines at frontier labs or productized for security & compliance use cases. Definitely warrants more sophisticated follow-on research.

    Read full reviewShow less

  2. Nice work, extending linear deception probes to VLMs is a useful step. I'd be especially excited to see future work push more towards strategic deception setups rather than explicit instructions, focusing on cases where the model has its own incentives/goals and chooses to deceive in multimodal contexts.

  3. Well-written introduction and research that would be highly relevant for DoD. I would like to see a clearer “big picture” in the intro about the consequences of intentional misrepresentation in high-risk settings, ideally with one concrete, impactful case study. Is there any work comparing accuracy and deception rates for locally hosted vs. cloud-based VLMs? Given that orgs bear more liability for local models, it would strengthen the motivation if you could argue (or show) that locally hosted systems are at higher risk because vendors can deploy more robust and frequently updated safety layers in the cloud.

    Prototype findings are promising. Good next step would be to identify one or two specific applications with accessible test data and test on a realistic dataset for that use case (for example, reading medical records, or counting assets from satellite imagery). Even if the dataset is still small, results would be more meaningful if they clearly map to a real high-risk VLM deployment scenario.

    Read full reviewShow less

Cite this project

@misc{miki2025opening,
  title = {{Opening Doors to Multimodal Deception}},
  author = {Takeshi Miki and Don Yin},
  year = {2025},
  month = nov,
  note = {Submitted to Defensive Acceleration Hackathon, an Apart Research Sprint},
  howpublished = {\url{https://apartresearch.com/sprints/projects/opening-doors-to-multimodal-deception-egr7}},
  url = {https://apartresearch.com/sprints/projects/opening-doors-to-multimodal-deception-egr7}
}

More from Defensive Acceleration Hackathon

All 84 entriesBrowse all projects

Build something like this at the next Sprint

AI Collusion Research Sprint · Oct 23 - 25, 2026