Steering Llama 2 via Contrastive Activation Addition

MATS 学员:

Nick Gabrieli, Nina Panickssery (née Rimsky), Julian Schulz, Meg Tong

作者:

Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, Alexander Matt Turner

引用次数

671 Citations

摘要:

We introduce Contrastive Activation Addition (CAA), an innovative method for steering language models by modifying their activations during forward passes. CAA computes"steering vectors"by averaging the difference in residual stream activations between pairs of positive and negative examples of a particular behavior, such as factual versus hallucinatory responses. During inference, these steering vectors are added at all token positions after the user's prompt with either a positive or negative coefficient, allowing precise control over the degree of the targeted behavior. We evaluate CAA's effectiveness on Llama 2 Chat using multiple-choice behavioral question datasets and open-ended generation tasks. We demonstrate that CAA significantly alters model behavior, is effective over and on top of traditional methods like finetuning and system prompt design, and minimally reduces capabilities. Moreover, we gain deeper insights into CAA's mechanisms by employing various activation space interpretation methods. CAA accurately steers model outputs and sheds light on how high-level concepts are represented in Large Language Models (LLMs).

近期研究

非大国冲突与 AI 风险

作者:

Kristina Kempkey

日期:

August 26, 2026

引用次数:

合成角色预训练:从第一个 token 开始对齐

作者:

Julian Minder

日期:

August 13, 2026

引用次数:

常见问题

什么是 MATS 项目?
项目持续多久?