Jargonautica
Security and Privacy

Content Moderation

Filtering what goes in and what comes out. It sits outside the model, which means it can be updated immediately when something new appears, unlike model behaviour, which requires retraining. Every deployment needs some version of this, and its calibration is a policy question rather than a technical one.

Technical view

Implemented as classifiers over input and output covering defined harm categories, with thresholds set per category and per deployment context. Output-side filtering adds latency and interacts badly with streaming. The unavoidable trade-off is between false positives, which block legitimate use, and false negatives. Multilingual and coded language degrade classifier performance, and moderation decisions should be logged for audit and appeal.


Part of AI Terminology Index, section: Security and Privacy.

How it connects

Content Moderation links to 10 other terms. The faint lines are links between those terms. See it in the full graph.

GuardrailsGuardrailsSafety ClassifierSafety ClassifierAccuracy, Precision and RecallAccuracy, Precision…StreamingStreamingAbuse MonitoringAbuse MonitoringLanguage CoverageLanguage CoverageChildren and AIChildren and AIMisinformation and DisinformationMisinformation and …Speech to SpeechSpeech to SpeechData LabourData LabourContent Moderation