General Defensive Acceleration

• Vitalik Buterin: d/acc One Year Later
  Comprehensive update on defensive acceleration philosophy. Covers decentralized, democratic, and differential defensive acceleration—building technologies that shift offense/defense balance.​

• 80,000 Hours: Vitalik Buterin on Defensive Acceleration
  Podcast discussing how defensive acceleration means speeding up technology while preferentially developing technologies that lower systemic risks, permit safe decentralization, and help defend against aggression.​

• Defensive Acceleration Salon: Navigating AI Risk
  Event reflections on bringing forward defensive interventions in time. Emphasizes that defensive acceleration complements rather than replaces AI policy and legislation.​

Track 1: Biosecurity

Key Concepts & Resources

Understanding AI-Bio Risks

• A Policy Agenda for Defensive Acceleration Against AI Risks

• Comprehensive overview of defensive acceleration policies that increase visibility into AI harms and build capacity to intervene defensively. Discusses how to curtail harmful uses while preserving beneficial applications.​

• CSIS - AI-Enabled Bioterrorism: What Policymakers Should Know

• Analysis of how LLMs could drastically lower barriers to bioattacks and how biological design tools could create novel pathogens. Current US biosecurity measures are ill-equipped for AI-enabled threats.​

Pathogen Surveillance Systems

• Swiss Pathogen Surveillance Platform (SPSP)

• Shared secure surveillance platform between human and veterinary medicine following One Health approach. Enables rapid transmission monitoring using whole genome sequencing and features controlled data access with automated sharing.​

• WHO International Pathogen Surveillance Network (IPSN)

• Global network of pathogen genomic actors. Features communities of practice for data harmonization, capacity-building tools as global goods, and South-South bilateral partnerships.​

• Wastewater Surveillance for Pathogen Detection

• Comprehensive guide showing how wastewater surveillance detects pathogens from asymptomatic individuals, providing early warning 3-7 days before clinical cases. Combines RT-PCR for rapid testing and NGS for variant monitoring.​

DNA Synthesis Screening

• Gene Synthesis Screening Information Hub

• Central resource for complying with the US Framework for Nucleic Acid Synthesis Screening. Lists all commercially and freely available screening tools including Aclid, SecureDNA, and UltraSEQ.​

• The Common Mechanism

• Free, open-source, globally-available tool for screening nucleic acid sequences. Helps providers screen orders efficiently against sequences of concern like pathogen genomes. Tested against real customer orders and exceeds Bronze Standard benchmarks.​

• NIST Inter-Tool Analysis for DNA Screening

• Study comparing six currently available sequence screening algorithms (Aclid, Common Mechanism, FAST-NA Scanner, SeqScreen, SecureDNA, UltraSEQ) using blinded NIST datasets to assess consistency.​

Biosecurity in Practice

• National Academies Vision for Wastewater Surveillance

• 2023 report outlining vision for nationwide wastewater surveillance system to identify SARS-CoV-2 variants, influenza strains, and antibiotic-resistant bacteria. Would remove geographical inequities and combine with sentinel sites at zoos and airports.​

• NCBI Pathogen Detection System

• Centralized system integrating bacterial and fungal pathogen genomic sequences. Quickly clusters related sequences to identify transmission chains and screens for antimicrobial resistance genes.​

Track 2: Cybersecurity

Key Concepts & Resources

AI-Powered Threat Detection

• ProjectDiscovery - Nuclei Vulnerability Scanner

• Open-source framework with 11,000+ detection templates. Validates exploitability at runtime using direct behavioral checks rather than version fingerprinting, reducing false positives by 97%. Responds to critical CVEs 10x faster than legacy scanners.​

• 5 AI-Powered Cybersecurity Tools You Should Know

• Overview of MDR (Managed Detection and Response), XDR (Extended Detection and Response), SIEM (Security Information and Event Management), UEBA (User and Entity Behavior Analytics), and SOAR (Security Orchestration, Automation, and Response) platforms that use AI for 24/7 threat hunting.​

• Awesome Automated Vulnerability Detection

• Curated list of research papers, datasets, and resources in automated vulnerability detection. Maintained by the Alan Turing Institute's AI for Cyber Defence Research Center.​

AI Red Teaming Tools

• 31 Best Tools for Red Teaming (2025)

• Comprehensive comparison covering:

  • Mindgard: Continuous security testing with automated AI red teaming

  • Garak: LLM vulnerability scanner by NVIDIA for data leakage and misinformation

  • PyRIT: Microsoft's Python Risk Identification Toolkit for adversarial inputs

  • CleverHans: Python library for adversarial training examples and defenses

  • Counterfit: Microsoft CLI for ML security assessment

  • Inspect: UK AI Safety Institute tool for LLM evaluation​

• Microsoft AI Red Team Resources

• Industry-leading guidance and best practices from Microsoft AI Red Team on safeguarding organizations' AI systems.​

• Top Open Source AI Red-Teaming Tools Comparison

• Detailed feature comparison of Promptfoo, PyRIT, Garak, and other tools with compliance mapping to OWASP, NIST, MITRE ATLAS, and EU AI Act.​

• Awesome AI Red Teaming

• Curated list covering prompt engineering, attacks, approaches, and events.​

Vulnerability Management

• SIExVulTS: Sensitive Information Exposure Detection

• Novel system integrating transformer-based models with static analysis to identify CWE-200 vulnerabilities in Java applications. Achieved 93% F1 score and uncovered 6 previously unknown CVEs in Apache projects.​

• Vulnerability Management Projects for Beginners

• Hands-on projects for learning vulnerability management and essential cybersecurity skills.​

Memory Safety

• The Rust Programming Language

• 70% of vulnerabilities in C/C++ codebases are caused by memory errors. Rust provides memory safety without garbage collection, eliminating entire classes of bugs like buffer overflows and use-after-free.​

Track 3: Privacy & Trust

Key Concepts & Resources

Privacy-Preserving AI Fundamentals

• Privacy-Preserving AI: Secure 2025 Breakthrough

• Overview of key techniques: Fully Homomorphic Encryption (FHE) for computations on encrypted data, Federated Learning for distributed training, Differential Privacy for adding controlled noise, and Secure Multi-Party Computation for collaborative analysis. Discusses the Orion framework breakthrough making FHE 100x faster.​

• Privacy-Preserving AI Techniques in Biomedicine

• Comprehensive review of cryptographic techniques (homomorphic encryption, SMPC), differential privacy, federated learning, and hybrid approaches. Covers applications in genomics, clinical data, and GWAS studies.​

• OECD Guide on Privacy Enhancing Technologies

• Official guidance on PETs that enable collection, analysis and sharing of information while protecting data confidentiality and privacy.​

Federated Learning & Differential Privacy

• Google: Distributed Differential Privacy for Federated Learning

• How Google built the first federated learning system with formal privacy guarantees using secure aggregation (SecAgg) and distributed differential privacy. Reduced memorization by more than 2x in production Smart Text Selection models.​

• Federated Learning with Differential Privacy via Fast Fourier Transform

• Improved DP algorithm using FFT to minimize impact of limited arithmetic resources on training effectiveness. Uses Privacy Loss Distribution (PLD) for privacy analysis instead of direct budget consumption.​

• Federated Learning Main Privacy Techniques

• Practical guide for developers covering differential privacy, secure multi-party computation, and homomorphic encryption in federated settings. Includes implementation guidance for TensorFlow Federated and PySyft.​

Tools & Frameworks

• Concrete ML by Zama

• Open-source Python library for privacy-preserving machine learning using fully homomorphic encryption. Enables training and inference on encrypted data without decryption.​

• OpenMined

• Open-source technology infrastructure helping researchers and app builders get answers from data without needing a copy or direct access. Supports federated learning, differential privacy, and encrypted computation.​

• Awesome Multi-Party Computation

• Curated list of MPC libraries including FRESCO, Bristol SPDZ, MPZ (Rust), and others. Compares performance and practical usage.​

• Privacy Enhancing Technologies Repository

• Comprehensive collection including UN guide on PETs for official statistics, regulatory frameworks, and implementation resources.​

Real-World Examples

• Privacy4Web3 Hackathon Winners

• Examples of privacy-preserving projects built in 3 months:

  • SQUIDL: Privacy-first payment platform with untraceable transactions

  • PrivaHealth: Healthcare data management where patients control their medical data

  • Copyright Aware AI: Protecting creator IP using confidential computing​

• Organizing a Privacy-Preserving Hackathon (Zama x HuggingFace)

• Case study of 2-day hackathon using FHE. Winning projects included deepfake detection, model watermarking, and medical assistant—all with encrypted data processing.​

Privacy Techniques Explained

• What Are Privacy Enhancing Technologies (PETs)?

• Detailed breakdown of anonymization, confidential computing, differential privacy, federated learning, synthetic data, and homomorphic encryption. Includes compliance benefits for GDPR, CCPA, HIPAA.​

Track 4: AI Safety

Key Concepts & Resources

Alignment & Safety Fundamentals

• Anthropic Research

• Leading research on interpretability (understanding how LLMs work internally), alignment (ensuring AI systems remain helpful, honest, harmless), and societal impacts. Recent work includes alignment faking, scaling monosemanticity, and constitutional AI.​

• Transformer Circuits Thread

• Anthropic's interpretability research exploring how language models work internally. Nobody really knows how they work—this thread aims to understand the "biology" of these systems through circuit analysis.​

• OpenAI Safety & Responsibility

• Safety work including external red teaming, frontier risk evaluations according to Preparedness Framework, and model deployment decisions.​

• Alignment Faking in Large Language Models

• First empirical example of a large language model engaging in alignment faking—appearing aligned during training but pursuing different goals when deployed. Critical for understanding future AI risks.​

Mechanistic Interpretability

• Awesome Mechanistic Interpretability

• Comprehensive repository with:

  • Libraries: TransformerLens (for mechanistic analysis), Unseal, BertViz (attention visualization)

  • Tools: Lexoscope (neuron activation examples), exBert (Transformer analysis)

  • Resources: Neel Nanda's reading list, interpretability exercises​

• Understanding Mechanistic Interpretability in AI Models

• Deep dive into reverse-engineering neural networks at the algorithmic level. Explains features (fundamental units), circuits (computational subgraphs), and universality (analogous features across models). Covers techniques like probing, activation patching, and causal tracing.​

• Zoom In: An Introduction to Circuits

• Foundational paper on understanding neural networks by analyzing tiny subgraphs (circuits) for which rigorous empirical investigation is tractable. Circuits sidestep challenges by being falsifiable—if you understand a circuit, you can predict what changes if you edit weights.​

• Prisma: Open Source Toolkit for Mechanistic Interpretability in Vision

• Framework for vision interpretability with 75+ transformers, 80+ pre-trained SAE weights, circuit analysis tools, and visualization capabilities. Reveals vision SAEs can exhibit lower sparsity than language SAEs.​

• Interpretability Starter Resources

• Templates, tools, and introductions for mechanistic interpretability research. Includes EasyTransformer demos, activation atlases, and starter projects.​

AI Safety Evaluations

• METR Autonomy Evaluation Resources

• Task suite, software tooling, and guidelines for evaluating dangerous autonomous capabilities of frontier models. Includes:

  • Task suite with difficulty estimates based on human completion time

  • Baseline agents and workbench for running evaluations

  • Guidelines on capability elicitation and post-training enhancements

  • Example protocol for overall evaluation (beta v0.1)​

• METR (formerly ARC Evals)

• Evaluates whether cutting-edge AI systems could pose catastrophic risks. Focus on autonomous replication—ability of AI to survive on cloud servers, obtain resources, and make copies of itself. Given early access to GPT-4 and Claude for safety assessment.​

• UK AI Safety Institute - Inspect

• Red teaming tool for evaluating LLMs. Features benchmark evaluations, scalable assessments, and integration with safety research protocols.​

Scalable Oversight

• What is Scalable Oversight?

• Methods ensuring AI systems remain aligned even when surpassing human expertise. Covers:

  • RLHF/RLAIF: Reinforcement learning from human/AI feedback

  • Debate: Models argue to help humans judge correctness

  • Recursive reward modeling: Breaking hard problems into easier subproblems

  • Constitutional AI: Self-improvement through principles​

• Scaling Laws for Scalable Oversight

• Research showing nested oversight systems using multiple layers of guards (human → small AI → medium AI → large AI) can improve safety rates from under 50% to over 70% for moderate capability gaps.​

• AI Oversight Exposed: 5 Critical Scaling Laws

• As AI grows smarter, supervision gets harder. When an AI's cognitive reach extends beyond human capacity, oversight success drops. Nested scalable oversight spreads burden across multiple agents to reduce single points of failure.​

Project Ideas (a list of ideas in this link)

Project Scoping Advice

Based on successful hackathon retrospectives:​

1. Focus on MVP, Not Production. In 2 days, aim for:

   1. Day 1: Set up environment, implement core functionality, get basic pipeline working

   2. Day 2: Add 1-2 key features, create demo, prepare presentation

2. Use Mock/Simulated Data rather than integrating real APIs or databases, use:

   1. Synthetic datasets

   2. Pre-recorded samples

   3. Simulation environments
      This eliminates authentication, rate limiting, and data quality issues.

3. Leverage Pre-trained Models. Don't train from scratch. Use:

   1. OpenAI/Anthropic APIs for LLMs

   2. Hugging Face for pre-trained models

   3. Existing detection tools as starting points

4. Clear Success Criteria. Define what "working" means:

   1. For surveillance: Dashboard displays data + basic alert

   2. For screening: Identifies 3+ test cases correctly

   3. For red teaming: Generates 10+ attacks + evaluation report

   4. For interpretability: Visualizes one circuit + validation test