Links
Focus
Alignment Training Methods, Interpretability, Agent Foundations
Alex is currently working on training invariants into model behavior. In the past, he formulated and proved the power-seeking theorems, co-formulated the shard theory of human value formation, and proposed the Attainable Utility Preservation approach to penalizing negative side effects.
Highlighted outputs from past streams:
- MATS 3.0: Mechanistic interpretability to understand and control maze-solving agents — introduced the now-staple technique of “steering vectors”
- MATS 3.1 extension: Steering GPT-2-XL by adding an activation vector
- MATS 4.0: Steering Llama-2 with contrastive activation additions
- MATS 5.0: Unsupervised discovery of model behaviors using steering vectors
- MATS 6.0: Gradient routing
- MATS 7.0: Unlearn and distill, for making robust unlearning a reality
- MATS 8.0: Output supervision can obfuscate the CoT
- MATS 9.0: Eval cooperativeness may be a scalable mitigation for eval gaming