Jailbreak
A jailbreak makes a model ignore its own safety training. It is related to prompt injection but not the same, and the difference decides who is harmed.
A jailbreak is an input crafted to make a model act outside the behavioural policy its provider trained into it — producing content it would normally refuse, adopting a persona that has no restrictions, or reasoning about a task it was aligned to decline.
How it differs from prompt injection
The two are constantly used interchangeably and they are not the same attack. The distinction that matters is whose policy is being broken and who is harmed.
| Jailbreak | Prompt injection | |
|---|---|---|
| Target | The model provider's safety policy | The application's instructions |
| Attacker | Usually the user themselves | Often a third party, via content the app retrieves |
| Victim | The provider, and the deploying brand | The application's own users and data |
| Typical goal | Get disallowed output | Get the application to act — call a tool, leak context, exfiltrate |
A user who talks a chatbot into writing something offensive has jailbroken it. An attacker who plants text in a document so that the next person's assistant emails them the conversation has performed an indirect prompt injection. The first is a content-safety and reputational problem; the second is a security incident with a data-loss consequence.
They overlap because the same techniques — role play, hypothetical framing, encoding, low-resource languages, many-shot conditioning — work on both, and a jailbreak is frequently the first step of an injection chain.
Why it is not fixable by refusal training
Alignment training shifts the probability that a model refuses; it does not install a boundary. Each new refusal behaviour is learned from examples, and the space of paraphrases is unbounded, so published jailbreaks are patched and replaced continuously. Treat provider-side safety as a useful default that raises the cost of casual abuse, not as a control you can point at in a risk register.
What to do about it
For most enterprise deployments, jailbreak resistance is the provider's problem and yours is the consequence: what the application does with the output. Log refusals and near-misses so you can tell whether you are being probed, keep the model on a task-scoped tool set so a bypassed refusal does not become an action, and treat every completion as untrusted input to whatever consumes it next.
The related risk classes are catalogued as OWASP LLM01:2025 Prompt Injection, which covers jailbreaking explicitly as a sub-case.
For the attack that actually reaches your data, read what prompt injection is and why filtering does not close it.