Jargonautica
Testing, Safeguarding and Running Models in Practice

Jailbreaking

Getting a model to bypass its own restrictions, usually by disguising the request. Common tricks include framing something as fiction, as a hypothetical, as a translation exercise, or as a task for a different persona. Every publicly available model has been jailbroken, and it remains an unsolved problem rather than a bug awaiting a patch.

Technical view

Techniques include role-play framing, encoding requests to evade input filters, gradual escalation across turns, many-shot approaches exploiting long context windows, and automatically generated adversarial suffixes. The underlying difficulty is that safety training shapes behaviour statistically rather than imposing hard constraints, so sufficiently unusual framings fall outside the distribution the training covered. Defence in depth, combining safety training with separate input and output filtering, is the standard practical response.


Part of AI Terminology Index, section: Testing, Safeguarding and Running Models in Practice.

How it connects

Jailbreaking links to 13 other terms, and the 12 closest are drawn here. The faint lines are links between those terms. See it in the full graph.

GuardrailsGuardrailsAlignmentAlignmentRed TeamingRed TeamingPrompt InjectionPrompt InjectionSystem PromptSystem PromptSafety ClassifierSafety ClassifierRefusal and Over-refusalRefusal and Over-re…System Prompt ExtractionSystem Prompt Extra…Adversarial ExamplesAdversarial ExamplesBug BountyBug BountyReinforcement Learning from Human FeedbackReinforcement Learn…Abuse MonitoringAbuse MonitoringJailbreaking