政策与治理
政策与治理方向研究先进 AI 目前如何治理，以及应该如何治理。随着 AI 能力快速提升，许多最棘手的问题已不再只是技术问题，还涉及国际协调、不确定性下的政策制定、国家能力、监管设计，以及如何将安全目标转化为现实政策与制度。未来 6 至 12 个月作出的决定，将影响实验室开发什么、政府提出哪些要求，以及未来数年的监督方式。
- 第一阶段：完成通用申请，并提交 1 至 2 份写作样本
- 第二阶段：回答研究流方向选择问题，并可能需要提供推荐人信息
- 第三阶段：参加面试和工作测试
本方向涵盖多类研究。有些研究流关注具体治理机制，包括评估、标准、安全措施、监控系统和执行机制；另一些则开展政策与制度分析，例如比较不同治理制度、监管框架和国际协调难题。还有一些研究流探讨更广泛的问题，例如先进 AI 如何改变全球权力格局，以及治理方式能否切实降低灾难性风险。
我们希望研究员能够清晰地分析和撰写相关议题。以往届别中的优秀申请者拥有多种背景，包括政策、经济、法律、政治学、公共管理、安全研究、哲学、计算机科学、预测研究、社会学、历史、新闻，以及科学与技术研究。
我们会根据契合度为研究员匹配导师，并规划项目，使其在项目结束前产出具体成果，例如政策备忘录、监管意见、技术规范、比较分析和同行评审研究。本方向的成果面向 AISI 工作人员、实验室治理团队、监管机构、标准组织，以及参与前沿 AI 治理的研究与政策社区。
TBD. Please check back later for details. Broadly speaking I'm open to a wide range of projects across AI policy and technical AI governance, and I'm open to pitches. Right now I'm sorting out some career plans that will affect the kinds of projects that I'd support.
My stream focuses on preserving checks and balances as governments adopt increasingly powerful AI. Fellows will work on questions like how Congress can maintain oversight of an AI-accelerated executive branch (including via privacy-preserving AI auditors) and what a positive vision for government AI adoption looks like. Projects will typically produce a public report and sometimes involve engaging directly with policymakers and other stakeholders.
We work arms-length with policymakers in Congress and the executive branch on timely, practical AI policy issues, while also conducting original research into future-oriented topics that may arise under plausible AI trajectories that have yet to enter the mainstream. High level themes include frontier AI governance, robotics / reindustrialization, and institutional disruption.
This stream focuses on identifying tractable policy and technical interventions to gradual disempowerment, focusing on economic disempowerment and the intelligence curse. Possible project areas include:
- Formalizing intelligence curse dynamics into a model that can be tracked and monitored.
- More durable policy solutions to mass unemployment than UBI.
- Technical interventions to extend the centaur period.
Escalation risks from state perceptions of AI capability, AI-enabled targeting, AI-enabled decision manipulation, and the impact of AI integration into nuclear command and control.
Our stream focuses on AI verification, as in how actors can check that the use of AI compute is compliant with policy, especially for enabling international agreements on AI. This sense of verification is much broader than formal verification.
This stream will focus on preparing AI governance policies for future policy windows through scenario mapping and policy architecture.
Research papers (technical governance or ML) related to evaluating and mitigating dangerous AI capabilities, with a focus on what's actionable and relevant for AGI companies