Richard Ngo
My MATS fellows will do philosophical thinking about multi-agent intelligence and how agents change their values. This will likely involve trying to explore and synthesize ideas from game theory, signaling theory, reinforcement learning, and other related domains.
Stream overview
I'd like theoretically-inclined scholars to work with me towards building up a theory of coalitional agency. This work will be highly abstract, philosophical and exploratory; it's mainly suited for people with strong mathematical backgrounds who've read some of my existing writings on this topic and find them interesting.
Mentors
Richard previously worked on alignment at DeepMind and governance at OpenAI. He's currently an independent researcher focusing on multi-agent intelligence. He's particular interested in understanding how subagents aggregate to form robust larger-scale agents, and how those larger-scale agents change the values of their subagents.
Mentorship style
I'll come meet scholars in person around 2 days a week on average. On those days I'll be broadly available for discussions and brainstorming. On other days scholars can message me for guidance (though I'd prefer to spend most of my effort on this during the in-person days).
Representative papers
See "Towards a scale-free theory of intelligent agency" and the various work linked from there.
Scholars we are looking for
My main criterion for selecting scholars will be clarity of reasoning.
You will probably will work with other scholars in the stream.
Project selection
I will talk through project ideas with the scholar.
Community at MATS
MATS Research phase provides scholars with a community of peers.
.webp)
During the Research phase, scholars work out of a shared office, have shared housing, and are supported by a full-time Community Manager.
Working in a community of independent researchers gives scholars easy access to future collaborators, a deeper understanding of other alignment agendas, and a social network in the alignment community.
Previous MATS cohorts included regular lightning talks, scholar-led study groups on mechanistic interpretability and linear algebra, and hackathons. Other impromptu office events included group-jailbreaking Bing chat and exchanging hundreds of anonymous compliment notes. Scholars organized social activities outside of work, including road trips to Yosemite, visits to San Francisco, and joining ACX meetups.