SolidGoldMagikarp (plus, prompt generation)

Anomalous tokens: a mysterious failure mode for GPT (which reliably insulted Matthew)

• We have found a set of anomalous tokens which result in a previously undocumented failure mode for GPT-2 and GPT-3 models. (The 'instruct' models “are particularly deranged” in this context, as janus has observed.)
• Many of these tokens reliably break determinism in the OpenAI GPT-3 playground at temperature 0 (which theoretically shouldn't happen).