Detection accuracy and false positives

Question 1

What is SecureAI Guard's detection rate?

Accepted Answer

We do not publish one, and we are not going to publish one we cannot reproduce. A detection rate is a property of a corpus, a threshold and a traffic distribution, not a property of a product — quote it without those three and it is unfalsifiable.

The corpus problem is not solvable by trying harder, either. Public prompt-injection corpora are the ones every detector in the category has already been tuned against, so a high score on them measures overlap with the training set rather than resistance to the attack. A number that would actually be worth publishing has to come from a corpus the detector has never seen, and that corpus is yours, not ours.

What exists instead is the method: guardrail evaluation methodology sets out the metric definitions, the corpus construction and the procedure in enough detail to be re-run against your own traffic and your own model. Use it against us and against every alternative you are looking at.

Question 2

What is the false-positive rate?

Accepted Answer

Same answer, and the arithmetic is why it matters more than the detection rate.

Attack traffic in a normal enterprise LLM application is rare. Suppose one request in a thousand is a genuine injection attempt — generous for most internal applications. Run a detector with a 1% false-positive rate over a million requests and you get roughly ten thousand false alarms against about a thousand true ones, before you count the true ones it missed. Nine out of every ten blocks your users experience are wrong.

Nothing about that ratio is a criticism of any particular detector. It is what base rates do to any classifier applied to a rare event, and it is the reason the only false-positive number worth having is one measured on your benign traffic, at the threshold you would actually deploy. A vendor's figure, measured on a balanced corpus where half the requests are attacks, describes a world you do not operate in.

Question 3

How do we measure this before enforcing anything?

Accepted Answer

Benign traffic first, in shadow mode. The order matters:

Run the layer in log-only mode over real production traffic for long enough to cover your weekly cycle. Every alert it raises in that window is a false positive by construction, because you have not been under a targeted campaign for the whole period — that is your false-positive rate at that threshold, on your distribution, for free. Read a sample of the alerts by hand. The rate matters less than the shape. Alerts clustered on one internal tool, one document type or one language are a tuning problem; alerts spread evenly across everything are a threshold problem. Only then build the attack corpus. Write it against your own application: your system prompt, your tools, your retrieved documents. Prompt injection explains why generic attack strings under-state the risk for a specific application — the interesting attacks name your tools. Hold part of it back. If you tune against the whole corpus, you have measured how well you tuned. Decide the enforcement threshold from the two curves together, and write down the false-positive rate you are accepting. A control deployed without a stated tolerance gets disabled the first time someone senior is blocked.

Question 4

Who deals with a block when it happens?

Accepted Answer

Decide before you enforce, because this is the operational cost people forget to budget. Every enforcing control needs an answer to: what does the end user see, can they retry, is there an appeal path, who sees the alert, and how long does a review take. A block with no route back is experienced as an outage regardless of how correct it was.

For higher-risk actions the answer is often not "block" at all but human-in-the-loop review — hold the action, surface it to a person, proceed on approval. That converts a false positive from a failure into a delay, which is a trade many teams should be making and few consider.

Question 5

Does a lower false-positive rate just mean a weaker threshold?

Accepted Answer

Frequently, yes, and it is the easiest number in the category to game. A detector tuned to almost never fire will report an excellent false-positive rate and catch correspondingly less. This is why the two numbers are meaningless apart and why the methodology insists on reporting them at a stated threshold, with the curve, rather than as a headline pair.

When a vendor quotes both, ask which threshold produced them, whether it is the default, and what the curve looks like either side of it. The answer tells you whether anyone there has run the measurement properly.

Question 6

Can prompt injection be detected reliably at all?

Accepted Answer

Not in the sense of "solved". Prompt injection is not a fixed signature set to be enumerated: it is any input that gets the model to treat data as instructions, and the search space is natural language. Detection reduces the rate; it does not close the class, and any vendor telling you otherwise is selling you a threshold.

The design conclusion is layering, not resignation. Assume some injections succeed and constrain what a successful one can reach — least-privilege tools, authorisation checked at the tool rather than inferred from the prompt, and output-side controls on what can leave. What guardrails can and cannot enforce and AI agent security both develop that argument, and it is the honest reason a detection number is not the thing to buy on.