Peter Henderson
Peter Henderson’s stream focuses on developing safe, aligned AI agents, with projects on scalable oversight rules informed by law and game theory, safe long-horizon exploration, and measuring “jagged” capability/safety frontiers. Scholars will join an independently driven, engineering-heavy research environment, collaborating with other MATS scholars and PhD students, with weekly 1:1s and active async mentorship.
Stream overview
I'd be interested in a variety of potential projects, but three directions that I'm excited by are:
- Building better AI agent rules for scalable oversight: In our recent work we showed that different models take different interpretation strategies when interpreting laws for AI. But there are still lots of things to figure out in making more complete specifications for AI agents. Can we leverage tools from law? What about reward specification and game theory?
- Safe long-horizon exploration for agents: As we create agents with the goal of discovery, the risks of reward mis-specification grow. How can we create good explorers with agency that nonetheless remain aligned over long horizons? Are there differential signs of misalignment in specific high-risk applications: AI R&D, bio research, cybersecurity research? How do we disentangle optimizing for long horizon tasks from implicit misaligned long-horizon goals (e.g., survival/agency versus task completion)? What if we want self-directed learning and exploration, how can we make this aligned in the long run?
- Measuring the jagged frontier: Jaggedness has been a problem, both for capabilities and safety. As the frontier becomes more jagged we might have seemingly highly capable models fail in surprising ways. How can we measure jaggedness of a model as a property? How can we leverage this in understanding safety and jailbreaking risks?
Mentors
Peter is an assistant professor at Princeton University, where he works on reinforcement learning, alignment, and law. He received a J.D. and Ph.D. in computer science from Stanford University.
Mentorship style
45 min weekly meetings by default for high-level guidance. I'm active on Slack for quick questions or conceptual (not code) debugging. Expect async back-and-forth on experiment design and results between meetings. Scholars can also schedule ad-hoc calls if they're stuck or want to brainstorm—just ping me on Slack. Other team members (PhD students) will also be around to help brainstorm, getting unstuck.
Fellows we are looking for
Essential:
- ML engineering experience—specifically implementing and training models on, e.g., slurm clusters.
- Research experience: at least one project where you drove the research direction (first-author or equal contribution).
- Familiarity with basic reinforcement learning principles and algorithms.
Nice to have, but not necessary:
- Prior RL papers.
- Theory background.
- Game theory background
- Law background
Not a good fit:
- Scholars who only have high-level ML knowledge without hands-on implementation
- Those who need heavily structured guidance (I will provide guidance and the goal is to work as part of a larger team, but will still need to independently drive research between team meetings and 1-1s).
Collaborators will be other MATS scholars, as well as PhD students/post-docs at multiple institutions (including Princeton).
Project selection
Mentors in the group will pitch projects, and scholars will try ones they find interesting for a week. We'll iterate together at the end of week 1 and pick final assignments in week 2.
The Winter 2027 cohort offers a wide range of research streams led by experts across AI alignment, interpretability, governance, and safety. Each stream provides its own research agenda, methodology, and mentorship focus.
