Shi Feng
The stream will focus on conceptual, empirical, and theoretical work on scalable oversight and control. This includes but is not limited to creating model organisms for specific failure modes, designing training procedures against them, and making progress on subproblems involved in safety cases.
Stream overview
- Realistic model organisms of deception and collusion
- Easy-to-hard generalization of monitors
- Legibility and easy-to-hard generalization of scalable oversight protocols
- Metacognition
High-touch (+2 hours of weekly 1:1s)
Washington, D.C.
Weak preference
Strong preference
Mentors
Shi Feng leads a research group working on oversight and control. He is an assistant professor at George Washington University. Prior to that, he was a postdoc in the NYU Alignment Research Group under Sam Bowman. He currently focuses on deception and collusion, with an emphasis on propensity and evaluation realism.
Fellows we are looking for
- Interested in hybrid (conceptual, empirical, theoretical) work on scalable oversight / control methods and threat modeling against them.
- The ability to articulate the crux of a (proposed) work and translate that into empirical experiments: what are the hidden assumptions? what hypothetical finding makes the idea more or less promising?
- The ability to look at the data and do careful manual qualitative analysis, e.g., reading debate transcripts.
- Comfortable with building scaffolds and various post-training processes.
- Human evaluation experience is a plus.
Project selection
A research agenda document will be shared ahead of time with a short list of project ideas. The scholars can also brainstorm and pitch ideas that are aligned with the research agenda. We will decide on assignments in week 2.