Google Vertex AI and inline LLM security

Google's own generative-AI SDK documents the attachment point this page is about, by name and for exactly this purpose. That makes Vertex AI one of the more straightforward platforms to put a security layer in front of — and makes the interesting question not "can I" but "what still gets past it".

We publish no tested-compatibility claim for Vertex AI. Our own tested support belongs in the supported models and frameworks matrix, which is not filled in yet. Everything below is either documented by Google and linked, or follows from where a control sits.

The documented attachment point: a custom base URL

The google-genai Python SDK supports overriding the base URL through HttpOptions, and the documentation gives the reason in the same breath: you "might need a custom base url (for example, API gateway proxy server) and bypass some authentication checks for project, location, or API key". The documented shape is:

base_url = 'https://test-api-gateway-proxy.com'
client = Client(
    enterprise=True,
    http_options={
        'base_url': base_url,
        'headers': {'Authorization': 'Bearer test_token'},
    },
)

HttpOptions also carries api_version, so a gateway can be pinned to the API version it was written against rather than following whatever the SDK defaults to.

Source: google-genai Python SDK documentation, read 13 August 2026. Check it before you build; SDK options move.

What that pattern enforces

Pointing the client at a layer you run puts a real enforcement point in the request path, out of process, with the properties that follow from that position:

• Every call through that client is covered, whichever part of the application made it and whether or not it went through an agent framework.
• It sees both directions — the assembled request including retrieved context and tool definitions, and the completion before the application acts on it. Input-side inspection alone cannot enforce output handling.
• It is server-side, so it cannot be removed by modifying a client, which is the difference between a control and a suggestion.
• Credentials stay yours. A gateway that forwards to Vertex with your service-account credentials means the application never holds them, which is a useful secondary benefit that often justifies the hop on its own.

What routes around it

Be precise about the boundary, because a base-URL override is a per-client-instance setting and nothing enforces that every client is constructed that way:

• Another SDK, or plain REST. A service using a different client library, a script, a notebook, or a curl in a cron job reaches Vertex directly. The durable fix is not documentation, it is network policy: allow egress to the model API only from the layer, so a client configured wrongly fails loudly instead of quietly bypassing the control.
• Anything that is not a model call. Retrieval, tool execution and index writes do not pass through the model API at all. Authorisation for those is enforced at the retriever and the tool — see RAG security and AI agent security.
• Streaming subtleties. If your routes stream, decide whether the layer buffers the response before releasing it or inspects it incrementally. There is no third option and the trade is real; it is worked through in latency and performance.

Platform safety settings are not the same control

Vertex AI provides its own configurable safety settings on generation requests, and they are a sensible baseline. They are also a different job from the one on this page: they are model-behaviour controls, tuned to categories the platform defines, applied to the content the model produces.

They do not know your data classification, your tool permissions, your users or your retention policy — and a policy expressed only in a platform's own settings does not travel when a team adds a second provider. Treat platform safety settings and an application-level security layer as complementary, and make sure your own diagram says which one is responsible for what. Reading Google's current documentation for the exact categories and thresholds is worth an hour before you assume coverage.

Region, residency and the extra hop

Vertex AI is a regional service, and the region you call is part of your transfer analysis rather than a deployment detail. Adding a security layer adds a hop, and if that hop is in a different jurisdiction from the model endpoint, you have changed the answer to "where is this processed" without changing anything on the architecture diagram.

Check Google's own locations documentation for what a given region commits to, and keep the layer in the same region as the endpoint unless you have a reason not to. GDPR and LLM data handling covers what the analysis has to establish; data residency records what is still open about our own hosted service; and on-premises deployment is the answer when the requirement is that content never leaves your boundary at all — no region selection satisfies that one.

What to ask us before you commit

• Do you support a base-URL gateway deployment in front of Vertex AI today, and for which SDKs and API versions?
• Can it be deployed into our own Google Cloud project and region?
• How does it behave on streaming responses?
• What happens to a call when the layer cannot reach a decision?

Evaluation and security review has the rest of the list.

Related

• The Anthropic API
• Amazon Bedrock
• Securing LangChain applications