Latest in AI
LiteLLM API Key Exposure: What to Rotate and What a Gateway Budget Cannot Stop
A response checklist for teams whose model gateway may have exposed upstream credentials, with separate checks for proxy access and provider billing.
If a LiteLLM gateway exposes an upstream model-provider key, changing the gateway password does not revoke that key. Treat proxy access and provider access as separate problems, then verify both before calling the incident contained.
That distinction matters in light of Amazon's November 13 threat report. Amazon captured agent records describing attempts to extract provider credentials from exposed AI applications, including LiteLLM and Dify. The records are evidence of attacker intent and reported activity, not proof that every claimed compromise succeeded. Amazon explicitly says it could not verify outcomes outside its observation.
This is a defensive planning guide, not a claim that your deployment is affected or a test of those products.
Separate the three credential layers
Start with a small inventory. A gateway can hold credentials that mean very different things, even when they all look like API keys.
| Credential | What it controls | Where containment belongs |
|---|---|---|
| Gateway administrator credential | Gateway management access | Gateway authentication and administrator controls |
| Gateway virtual key | Proxy access, including any management permissions it carries | Gateway key management |
| Upstream provider credential | Access to the model provider | The issuing provider's account and key controls |
LiteLLM's virtual-key documentation distinguishes its proxy administrator master key from generated keys. It also documents spend tracking and key-level model access. Those are useful controls, but they operate inside the gateway's request path. A generated key is not necessarily a low-privilege client key: the documentation says an admin-owned key can reach management routes. Inspect its effective permissions rather than classifying it by name alone.
Consider a support application with a $100 gateway budget. If someone obtains its virtual key, the proxy still mediates their requests. If they obtain the upstream credential instead and use it directly with the provider, the proxy is absent from that path. A gateway budget cannot govern traffic the gateway never sees. This is an architectural distinction, not an observed incident in that hypothetical application.
Contain exposure before restoring service
Amazon's report points to familiar weaknesses: exposed management interfaces, default credentials, unfinished installations and unpatched applications. It does not establish one universal vulnerability or one universal patched version across all named products.
For an environment you administer, identify which service was exposed and preserve relevant logs while restricting access through your incident-response process. Record the deployment version, affected credential identifiers, known consumers and the earliest plausible exposure time. Do not paste full secrets into tickets or chat to make that inventory easier.
Then work through each credential layer. Revoking a client virtual key does not replace a compromised administrator credential. Replacing the administrator credential does not revoke upstream credentials already copied elsewhere. Do not delay revoking a suspected exposed provider key merely to finish the inventory. Coordinate application updates through the incident process, accepting an interruption where necessary rather than quietly leaving an old secret active.
Check provider usage, not just proxy logs
OpenAI's API-key safety guidance recommends immediate rotation when a leak is suspected and reviewing account usage. It also distinguishes spend alerts from hard enforcement: alerts alone do not stop traffic, while enforced limits can interrupt legitimate workloads and are not instantaneous.
That is provider-specific guidance, not a promise that every provider offers identical controls. For each upstream account, verify the available permissions, network restrictions and billing enforcement instead of assuming a setting in one console covers the others.
In the support-app example, compare provider usage with the gateway's records over the same time range and account scope. A discrepancy is a reason to investigate, not proof of theft: other legitimate applications, delayed reporting and different accounting windows can also explain it. Keep evidence that lets the account owner distinguish those possibilities.
Define what recovery must demonstrate
Before reconnecting production traffic, write down the acceptance checks. I would require confirmation that the exposed entry point is restricted, affected old credentials are revoked at their issuing layer, authorized applications use replacements, and provider usage has an owner watching it.
These are proposed checks, not tests performed for this article. Use non-destructive checks on systems you own, and do not test leaked keys against someone else's account.
The practical lesson is to make the gateway's upstream dependency map part of normal operations. If nobody knows which provider accounts sit behind a shared proxy, even a quick password change can leave the expensive part of the exposure untouched.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help technical teams explain security changes and developer workflows clearly, without turning them into marketing claims.