MATS Winter 2027

This stream is primarily focused on research into physical defenses against engineered pathogens, aiming to inform decisions about PPE stockpiling and distribution approaches, improve improvised PPE and bioshelter scale-up, and reach rapid conclusions on how much to prioritize other areas of physical biodefense (agriculture, emergency response, etc.).  We are also open to strategic research into the use of bioweapons by AI or AI-human teams as part of takeover strategies and how this might inform preparedness.

This coalition of mentors make up the “Anthropic Stream”. This stream spans a range of empirical research areas in AI safety on LLMs, including AI control, scalable oversight, model organisms, model internals, model welfare, security, and more. You’ll be pitched, and have the option to pitch, a variety of safety research projects, and then be matched to projects and mentors based on your interests/preferences on research and what you’d like to get out of MATS. Fellows in this stream frequently receive funding and continued mentorship after MATS to complete their research project, usually leading to a (co-)first author paper. People in this stream often end up in long-term homes for safety research after MATS (e.g. Anthropic, Redwood Research, OpenAI).

​

Anthropic mentors share an application, tend to collaborate and co-mentor projects together, and generally share infrastructure to streamline the fellow experience. By applying to this stream, you are being considered for all of the Anthropic mentors.

We will continue working on black-box monitors for scheming in complex agentic settings, building on the success of the previous stream. Concretely, we will work on scaling our datasets and fine-tuning efforts, as described in the scalable monitoring agenda

​

Most likely the next projects will be about automated iterated red-team vs. blue-team games. We are currently training the blue team. We will then train the red-team and within this stream, we will try and close the loop to train them both synchronously.

This stream focuses on building a Science of Scheming, i.e. what are the mechanisms by which future models might become schemers, even though current models are not. We want to discover empirical Scaling Trends for Scheming. For example, does deceptive alignment become easier to discover with improved model capabilities?

Theory of change: Soon, most important work will be done by AI. AI is going to increasingly advise people and help with important things, many of which are time-sensitive and path dependent, e.g., work on alignment/safety (including various things like how LLMs should behave given that they’re very persuasive); how to think about acausal trade; how to organize society. It seems good for AI to do well at those things.

​

Of course, a lot of the relevant skills for doing well at these tasks are the same skills that cause AI risk and that AI companies work on (and are incentivized to work on) by default; like coding, some kinds of forecasting, etc.

​

We want to make models better at things that are net positive for the future, but that likely won’t benefit much from said default training (or perhaps will even be made worse by such training – e.g., via sycophancy).

​

In practice, a lot of the tasks that we’re interested in from this perspective are what we call “conceptual”: tasks that are hard to verify and don't have clear ground truth but where we nonetheless feel like we can make progress through argument and reason.

​

You can visit conceptualreasoning.ai to get a sense of our work to date.

​

We also take a keen interest in projects directly aimed at making future acausal interactions go well.

We aim to generalize tools for analyzing the dynamics of large-scale agency and power, such as public choice theory, to the setting in which machine minds are competitive with humans.

I currently lead the science of evaluation team at the AI Security Institute in London. I'm interested in topics around dangerous capability measurement, and understanding agent behaviours and goals, and their implications for policy.

This stream focuses on how advanced AI could enable new and dangerous bio technologies, and on assessing when risks become tractable or urgent as those capabilities arrive.