this is a great hackathon project (well done, Lukas & Axel). it proposes a tangible/scalable product beyond evals/consulting (guardrail models) and does so for a well-defined ICP & industry (CISOs/CTOs/etc at robotics companies). robotics is a great niche/wedge re vertical safety products like this for markets with particularly high bars for trust/safety (robotics, healthcare, finance, etc).

This is a much-needed idea - as you point out, AI safety work related to robotics has largely been neglected. You explain the problem well and it's great to hear you already have momentum. Well done! However, I am unsure about how scalable this is - as mentioned for large robotics companies safety is likely best implemented in an integrated way within internal production processes. Perhaps consider whether there is a way of de-risking this aspect of your project?

The problem is real and significant*. Multimodal evals are also a relatively underexplored area, even though consequences are significant.

Usually the limitation with these kinds of ideas is that they're essentially solving an extremely large surface area that a big lab is incentivised to work on themselves end-to-end (it's hard to make a useful robot that also randomly injures people so someone's usually working on Don't Make Your Robot Kill People), whereas for B2B what you almost always want is to solve something specific that removes the need to hire 1-2 people. That said, the problem descriptions are well-done and show an attention to detail and ability to iterate with real examples, so this is promising.

Perfect score would be granted for a clearer path to monetisation, either solving something the big labs would find hard to solve themselves, or providing a service that makes sense to outsource/use third party for. But tech-wise, this is pretty sound.

I actually know someone working on Dangerous Capabilities Evals at OpenAI who I pitched multimodal safety evals to, if you'd like an introduction :)

*I am biased because I work on multimodal evals and interpretability

It seems like you have understood the problem space very well and are uniquely positioned to solve this problem (Great job on this). I think its fair to put emphasis on the problem and go with a very simple approach as of now, though most effort will obviously lie in experimenting with an actual ROS/LLM.