SecureBio AI

该研究流将通过实证项目评估 AI 滥用带来的国家安全威胁(包括化学、生物、放射性和核恐怖主义及网络攻击),并改进危险能力评估。威胁建模方向希望申请者保持审慎、乐于开展案例分析,并具备出色的书面沟通能力。评估方向希望申请者能够积极参与,帮助验证一些概念,例如在 AI 滥用情境下模型是否存在“藏拙”评估缺口。

研究小组概览

This stream is primarily interested in mentoring projects in biosecurity that either (1) create rigorous threat models of AI biological misuse or (2) create benchmarks and tools that allow us to evaluate and mitigate these risks, as well as verifying that companies are taking suitable precautions.

​

Potential example projects include:

  • Threat Model: What do inference compute trends imply for how fast dangerous biological capabilities may proliferate and become harder to monitor?
  • Evaluations: Formalizing "scientific ideation in an empirical field" in a manner that allows one to assess human and LLM-generated hypotheses for novelty, plausibility, etc.
  • Mitigations: Developing a way to more richly assess and describe the "blast radius" or "collateral damage" of efforts to remove-in-pretraining or unlearn material from LLMs
  • Verification: How are we better able to standardize and compare the effectiveness of classifiers from different AI companies and assess how much they reduce misuse risk?

导师

Jasper Götting
SecureBio
,
Head of AI
Biosecurity
Capability and Propensity Evaluations

我在 SecureBio 担任 AI 研究负责人,专注于生物安全领域的 AI 评估。此前,我在 Convergent Research 担任研究员,从事 far-UVC 空气消毒研究,并在 Hannover Medical School 获得病毒学博士学位。

工作之余,我喜欢阅读、主持桌上角色扮演游戏、拍云、徒步和打鼓。

阅读更多

我们希望招收的学员

For threat modeling work:

  • Skeptical mindset
  • Transparent reasoning
  • Analytical

​

For evaluations, mitigations, and verification work:

  • LLM engineering skills (e.g., agent orchestration)
  • Biosecurity knowledge

项目遴选

导师会与学员讨论项目想法。