Latency and performance questions

We publish no latency figure for SecureAI Guard, and this page is the reason plus the method. A single number quoted without the payload distribution, the placement and the decision path that produced it is not a measurement, it is a marketing artefact — and you can get a better one for your own traffic in an afternoon.

How much latency does an inline security layer add?

It depends on four things, and any vendor number that does not state all four is unusable:

1. Whether the decision is in the request path at all. Enforcement is synchronous by definition; monitoring can be asynchronous and costs the request nothing.
2. How the decision is made. A pattern or classifier check is sub-millisecond to low-milliseconds work. A check that itself calls a model costs a second model round trip, which usually dominates everything else on this list.
3. Payload size. Inspection is work per token of context. A RAG request carrying 40 KB of retrieved chunks is not the same measurement as a 200-byte chat turn, and production traffic is mostly the former.
4. Network placement. Same host, same VPC, same region, or across a region boundary — this alone can move the added latency by an order of magnitude and has nothing to do with the product.

Platforms that report this are explicit about it: Amazon Bedrock's ApplyGuardrail response includes a guardrailProcessingLatency field in its invocation metrics, precisely because the number is per-call and per-configuration rather than a property of the product (AWS documentation, read 13 August 2026).

Why does this site not publish a benchmark figure?

Because we have not run a measurement we would be willing to defend, and publishing one anyway would refute the position we take everywhere else on this site. Our own research methodology argues that a number shipped without its corpus, its metric definitions and its procedure should be rejected by the reader. Stubbing a plausible figure here to fill the gap would make that page a piece of theatre.

The honest statement is: no published figure, a documented method for producing one, and an invitation to run it against your own traffic during an evaluation. Hold every other vendor in the category to the same standard, and notice how many published numbers survive the question "on what corpus".

What is the hard case — streaming responses?

Yes, and it is the question most evaluations forget to ask. When the model streams tokens back, output-side inspection has two options and neither is free:

• Buffer the response, inspect it, then release it. The check is complete and nothing unsafe is ever shown, but time-to-first-token becomes time-to-last-token. For a chat interface that is the difference between a product that feels instant and one that feels broken, even though total latency barely moved.
• Inspect incrementally as tokens arrive. Perceived latency is preserved, but a decision can only be made on what has arrived so far, and anything already released to the client cannot be recalled. A leak that completes in the first forty tokens is out before the check that would have caught it can see the rest.

There is no third option, and the choice is a policy decision rather than a technical one: it depends on whether the control is protecting against embarrassment or against disclosure. Ask any vendor which of the two their product does, whether it is configurable per route, and what happens to a partial response that is blocked mid-stream. This site does not publish SecureAI Guard's behaviour here, so ask us the same question and get the answer in writing.

How do we measure the overhead ourselves?

Measure at your application boundary, not at the vendor's. The procedure is short:

1. Capture a sample of your real production requests — at least a few hundred, sampled across routes, so the payload-size distribution is yours and not a synthetic one.
2. Replay them against the model directly, recording time to first token and time to completion at the p50, p95 and p99. This is your baseline, and it is the number that is missing from most evaluations.
3. Replay the same sample through the security layer with the policy you would actually deploy — not a permissive demo policy, which does less work.
4. Report the difference at each percentile, and report it separately for streaming and non-streaming routes. The p99 is the one that generates tickets; the p50 is the one that gets quoted.
5. Repeat with the layer placed where it will really run. A benchmark taken on the same host as the application is not evidence about a cross-region deployment.

Nothing in that list is specific to this product. It is the same procedure that makes the guardrail evaluation methodology reproducible, and it produces a number you can defend to your own platform team, which a vendor's number never will be.

Does the security layer become part of our availability budget?

If it is inline and synchronous, yes. That is the cost of enforcement being real, and it belongs in the design review rather than in the first incident. Decide the failure behaviour deliberately — see what happens if the layer is unavailable — set a timeout you have actually tested, and make sure the timeout path is exercised in load testing rather than discovered in production.

Can any of this run out of the request path?

Monitoring can, and often should. Detection that feeds a dashboard, an alert or a review queue does not need to block the response, and moving it out of the synchronous path removes it from the latency conversation entirely. What cannot move out of the path is enforcement: a control that decides after the response was returned is a report, not a guardrail. What guardrails can and cannot enforce sets out that line in more detail.

The practical design is usually both — a small synchronous set of checks that must block, and a larger asynchronous set that must be seen. Splitting them deliberately is how teams keep the p99 acceptable without quietly turning the control off.

Where to go next

• Detection accuracy and false positives — the other half of the evaluation, and the one that decides adoption.
• Deployment and architecture — where the layer sits and how it behaves when it cannot decide.
• Guardrail evaluation methodology — the full procedure, corpus definitions included.