Forethought
AI macrostrategy: strategic questions about how the transition to advanced AI will happen, and what we can do now to prepare for it.
Topics of interest include better futures, power concentration, takeoff speeds, deals with AIs, space governance, and acausal trade.
Stream overview
Forethought is a small research organisation working on how to navigate the transition to very advanced AI. We’re generalists from lots of different backgrounds, and are broadly interested in strategic questions about how this transition will happen, and what we can do now to prepare for it.
We work on topics like:
- How to make the future better (rather than just avoid extinction)
- Power concentration, especially the risk of AI-enabled coups
- Takeoff speeds and intelligence explosion dynamics
- How many AI projects would be good, and how they could be governed
- AI tools for existential security
We’d be interested in mentoring people on a range of AI macrostrategy questions, including:
- Any of the topics above
- Concrete visions for a positive post-AGI society, and which we should aim for
- Space governance and strategy
- The implications of acausal and simulation-based trade
- In what circumstances to hand power to superhuman AI
- Mapping multipolar outcomes and thinking about how good or bad they would be
- Whether and how we can pay misaligned AI to help us
Mentors
Fin Moorhouse is a researcher at Forethought. Previously he was a researcher at the Future of Humanity Institute and Longview Philanthropy, and studied philosophy at Cambridge.
Mia Taylor joined Forethought after three years as a Researcher and then Interim Research Director at the Center on Long-term Risk. She received a bachelor’s degree in Mathematics and Computer Science from Harvey Mudd College.
Tom Davidson is the author of a series of reports on AI timelines, and whether AI could drive explosive growth. Tom was previously a Senior Research Fellow at Open Philanthropy, a research scientist at the UK Government's AI Security Institute, and a data scientist at a startup. Tom has a first-class masters degree in physics and philosophy from the University of Oxford.
Mentorship style
Each scholar will be assigned a primary mentor who will meet with them once a week. The specifics will depend on the candidate and project.
Representative papers
Scholars we are looking for
We’re looking for people who:
- Write and think clearly
- Have good taste for which directions matter and are likely to bear fruit
- Have strong reasoning skills
- Are fairly independent, comfortable working autonomously and setting some of the direction themselves
- Are truth-seeking and open to changing their mind
- Care about the future going well
It’s a bonus if you already have research experience, or have domain knowledge in a relevant field like philosophy or economics.
Project selection
For project ideas, see here
Community at MATS
MATS Research phase provides scholars with a community of peers.
.webp)
During the Research phase, scholars work out of a shared office, have shared housing, and are supported by a full-time Community Manager.
Working in a community of independent researchers gives scholars easy access to future collaborators, a deeper understanding of other alignment agendas, and a social network in the alignment community.
Previous MATS cohorts included regular lightning talks, scholar-led study groups on mechanistic interpretability and linear algebra, and hackathons. Other impromptu office events included group-jailbreaking Bing chat and exchanging hundreds of anonymous compliment notes. Scholars organized social activities outside of work, including road trips to Yosemite, visits to San Francisco, and joining ACX meetups.