Verifying LLM Inference to Detect Model Weight Exfiltration

MATS 学员:

Roy Rinberg, Daniel Reuter, Adam Karvonen

作者:

Roy Rinberg, Adam Karvonen, Alexander Hoover, Daniel Reuter, Keri Warr

引用次数

2 Citations

摘要:

As large AI models become increasingly valuable assets, the risk of model weight exfiltration from inference servers grows accordingly. An attacker controlling an inference server may exfiltrate model weights by hiding them within ordinary model outputs, a strategy known as steganography. This work investigates how to verify model responses to defend against such attacks and, more broadly, to detect anomalous or buggy behavior during inference. We formalize model exfiltration as a security game, propose a verification framework that can provably mitigate steganographic exfiltration, and specify the trust assumptions associated with our scheme. To enable verification, we characterize valid sources of non-determinism in large language model inference and introduce two practical estimators for them. We evaluate our detection framework on several open-weight models ranging from 3B to 30B parameters. On MOE-Qwen-30B, our detector reduces exfiltratable information to <0.5% with false-positive rate of 0.01%, corresponding to a >200x slowdown for adversaries. Overall, this work further establishes a foundation for defending against model weight exfiltration and demonstrates that strong protection can be achieved with minimal additional cost to inference providers.

近期研究

非大国冲突与 AI 风险

作者:

Kristina Kempkey

日期:

八月 26, 2026

引用次数:

合成角色预训练:从第一个 token 开始对齐

作者:

Julian Minder

日期:

八月 13, 2026

引用次数:

常见问题

什么是 MATS 项目?
项目持续多久?