MATS Fellow:
Jessica Cooper (Rumbelow), Matthew Watkins
Authors:
Jessica Cooper, Matthew Watkins
Citations
Abstract:
Anomalous tokens: a mysterious failure mode for GPT (which reliably insulted Matthew).
We have found a set of anomalous tokens which result in a previously undocumented failure mode for GPT-2 and GPT-3 models. (The 'instruct' models “are particularly deranged” in this context, as janus has observed.)
Many of these tokens reliably break determinism in the OpenAI GPT-3 playground at temperature 0 (which theoretically shouldn't happen).
AI Researchers' Views on Automating AI R&D and Intelligence Explosions
Authors:
Severin Field
Date:
March 5, 2026
Citations:
0