Safe ai

Really like the concept. I am very bullish on the usecase, I believe AI insurance will become really big within the next years. Open questions/comments on my end: Insurance is a very special and highly regulated industry, you would need to have someone on the founding team, who knows about it, as winning is most likely about execution speed & capital. You would really like to have someone aligned doing this, as this new insurance player would have vast resources to hire statisticians and scientists to effectively do AI safety and AI risk work. The more technical and aligned the management is, the more deeply technical (and open/cooperative) the company would likely be. Would be cool if you could calculate with an example case of insurance premium and risk, of e.g. the cancer case and put it on the slide. Open question on competition: Why will normal insurance companies not be able to implement this (or how much time do you have to implement it before they do).

I appreciated the thorough exploration from the team of the business case, beyond the technology.

I would have appreciated a deeper exploration on which are the legal provisions that are currently being employed when deploying AI agents into production (which is already happening).

I encourage the team to focus the MVP even further, by choosing one particular segment or vertical, and start mapping all the existing commercially available AI agents which could be analyzed for safety and insurance policy.

The project shows some solid groundwork, especially in their methodology and pilot experiment planning - they've clearly thought through the implementation details and timeline, which is great. The demo looks promising too!

However, there are a couple of areas where it could be stronger. While they're targeting healthcare and legal AI, these fields are already pretty saturated with AI solutions - it would've been super helpful to see how they plan to differentiate from existing players in the market. This makes it a bit tricky to see how their research would scale into something truly impactful.

Also, from an AI safety perspective, the focus seems pretty narrow - they haven't really connected the dots between their technical approach and broader AI risk challenges..particularly x-risk. If they could expand their scope beyond just healthcare/legal and show how their risk assessment platform addresses larger AI safety concerns, that would make the project much more compelling.

Focuses on managing financial downsides rather than making progress on AI safety - insurance payouts don't prevent catastrophic outcomes. The timeline seems too optimistic, and overly focused on profitability. It's also unclear how they estimate the risk of the agents.