David Lindner

This stream will focus on monitoring, stress-testing safety methods, and evals, with a focus on risks from scheming AIs. Examples include (black-box) AI control techniques, white-box monitors (probes etc.), chain-of-thought monitoring/faithfulness, building evaluation environments, and stress-testing mitigations.

研究小组概览

I'm interested in detecting and mitigating deceptive alignment (mainly via capability evaluations and control). I'm interested in supervising projects in these areas:

  • Developing capability evaluations for deceptive reasoning ability.
  • Usefulness of externalized reasoning for oversight (CoT faithfulness by default, architectures promoting CoT faithfulness, monitoring of CoT)
  • Control (primarily simulating control evaluations in different settings).

项目期间的工作地点:

London

导师

David Lindner
Google DeepMind
,
研究科学家
Misalignment Science
AI Control and Monitoring
Capability and Propensity Evaluations
对齐训练方法

David Lindner 是 Google DeepMind 通用人工智能(AGI)安全与对齐团队的研究科学家,负责研究针对欺骗性对齐和密谋行为的评估与缓解方法。他近期的工作包括 MONA(一种减少强化学习过程中多轮奖励钻空子行为的方法)、设计针对隐蔽行为和情境意识的评估,以及参与制定 Google DeepMind 对欺骗性对齐的应对方案。目前,他关注密谋行为的缓解方法,包括思维链监控和 AI 控制。更多信息见个人网站。

阅读更多

我们希望招收的学员

I am looking for scholars with strong machine learning engineering skills, as well as a background in technical research. While I’ll provide weekly guidance on research, I expect scholars to be able to run experiments and decide on low-level details fairly independently most of the time. I’ll propose concrete projects to choose from, so you should not expect to work on your own research idea during MATS. I strongly encourage collaboration within the stream, so you should expect to work in teams of 2-3 scholars on a project, hence good communication and team skills are important.

项目遴选

We will most likely have a joint project selection phase, where we present a list of projects (with the option for scholars to iterate on them). Afterward, each project will have at least one main mentor, but we might also co-mentor some projects.