A separate, usually smaller AI model that screens what goes into and comes out of a main model, flagging or blocking harmful content. Rather than relying only on the main model to refuse bad requests, a classifier checks each message against a list of disallowed categories, like one filter at the door and another at the exit. Classifiers catch much that would otherwise slip through, but they can also block innocent requests by mistake.
Technical view
Input classifiers screen prompts and output classifiers screen completions, sometimes during streaming so generation can be halted part way. They are trained on labelled or synthetic examples against a written policy, as with Meta’s Llama Guard (December 2023), which uses a language model as the classifier, and Anthropic’s Constitutional Classifiers (February 2025), trained on synthetic data generated from a list of allowed and disallowed content. The trade-offs are false positives, which raise over-refusal, extra compute and latency, and adaptive attackers who probe for blind spots. Being independent of the main model, they add defence in depth and can be updated without retraining it.
Related
Part of AI Terminology Index, section: Testing, Safeguarding and Running Models in Practice.