OCAP Agents
Dmitrii Volkov · Team Little stars ✨
Submitted to Agent Security Hackathon. Sprint projects are early-stage work by participants, not Apart Research publications.
Building agents requires balancing containment and generality: for example, an agent with unconstrained bash access is general, but potentially unsafe, while an agent with few specialized narrow tools is safe, but limited. We propose OCAP Agents, a framework for hierarchical containment. We adapt the well-studied paradigm of object capabilities to agent security to achieve cheap auditable resource control.
Reviews
This is a good approach to address potential misuse. However, I would have appreciated seeing some code implementation. Building even a basic system often reveals various trade-offs we might not anticipate otherwise. I'm interested in seeing how this idea could be extended and applied in real-world scenarios.
• I really liked the concept “Separation of "controller" and "worker" agents” having top-level agents for refinement while having sub-agents with narrower tasks and fewer permissions. Reminded me to AutoGPT but with a target in security.
• It would have been very useful seeing an MVP in a quick notebook. Or at least a justification of why there is no code submission.
• Since this is an idea only submission it would have been great to see some of the following: more info about the security tax of this implementation (e.g. by having this modular design you might increase latency, …), more developed initial thoughts about how to implement this if given enough time (e.g. tech stack, … although the IPC comms and kernel separation and great start points),…
Hierarchical agents with limited capabilities and permissions is a great idea. An example use case of email writing process makes sense to me. It would be a good to see a working demo on one potential “security” usecase in the future
This work gives a vision of agents existing within a capability management framework and gives convincing examples. While this is limited to subagents and does pass the buck to some degree, I think it’s a solid contribution and outlines a promising engineering direction..
Cite this project
@misc{volkov2024ocap,
title = {{OCAP Agents}},
author = {Dmitrii Volkov},
year = {2024},
month = oct,
note = {Submitted to Agent Security Hackathon, an Apart Research Sprint},
howpublished = {\url{https://apartresearch.com/sprints/projects/ocap-agents}},
url = {https://apartresearch.com/sprints/projects/ocap-agents}
}More from Agent Security Hackathon
- 1st place by peer reviewView project: Diamonds are Not All You Need
Diamonds are Not All You Need
Diamonds are Not All You Need
This project tests an AI agent in a straightforward alignment problem. The agent is given creative freedom within a Minecraft world and is tasked with transforming a 100x100 radius of the world into diamond. It is …
- View project: Cross-model surveillance for emails handling
Cross-model surveillance for emails handling
Fluffy Vin
A system that implements cross-model security checks, where one AI agent (Agent A) interacts with another (Agent B) to ensure that potentially harmful actions are caught and mitigated before they can be executed. …
- View project: Inference-Time Agent Security
Inference-Time Agent Security
Inference-Time Agent Security
We take a first step towards automating model building for symbolic checking (eg formal verification, PDDL) of LLM systems.