PARAS MADAN
All writing

Latest in AI

LiteLLM dev.4 Signature Verification: Replace the Key, Keep the Checks

LiteLLM changed its release-signing method, so a failing verification job needs a precise policy update rather than an exception.

LiteLLM’s 1.111.0-dev.4 prerelease uses keyless Cosign signing instead of the older public-key method. If your deployment pipeline still runs cosign verify --key, the release notes say that command will not verify this image. The fix is to check the expected signing identity and issuer, not skip verification.

The verification command for dev.4

The official dev.4 release supplies this command for its GitHub Container Registry image:

cosign verify \
  --certificate-identity https://github.com/BerriAI/project-releaser/.github/workflows/release.yml@refs/heads/main \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/berriai/litellm:v1.111.0-dev.4

The signing change began with dev.3. Earlier releases still use the public key. Keep the verification policy appropriate to the artifact you actually deploy, including an older artifact retained for rollback. A blanket replacement across every version would obscure that distinction.

This is a documented command, not a claim that I verified a downloaded image here. It also concerns a development prerelease, not a recommendation that production users abandon their current stable deployment.

Why both identity and issuer matter

Sigstore’s verification documentation describes identity-based verification using a certificate identity and an OpenID Connect issuer. For this release, the identity names a particular publishing workflow, while the issuer is GitHub Actions. The command deliberately specifies both.

Do not replace the exact workflow identity with a broad regular expression simply to get a green build. Accepting any identity from a familiar issuer would change what your deployment gate trusts. Likewise, a successful signature check establishes a particular provenance claim; it does not prove the application has no vulnerabilities or meets your production requirements.

Cosign also checks, by default, that its signature payload’s image digest matches the image being verified. Preserve that check. A signature-verification migration should not become an excuse to turn off unrelated validation.

Separate the dev.3 installation problem from signing

There is a second issue that can confuse this upgrade. The dev.3 release notice says it was only partly published and directs users to dev.4, built from the same commit.

Initially, installing litellm[proxy]==1.111.0.dev3 failed because two pinned dependencies were not yet on PyPI. The notice says those dependencies are now published, so that installation works. It separately records missing distributions in the enterprise package repository. Neither problem is evidence that turning off signature verification is appropriate.

Start by identifying the failed stage: package resolution, image retrieval or signature verification. A pip dependency error and a Cosign identity error need different investigations, even if they appeared during the same upgrade.

Change the pipeline without weakening it

Consider a deployment job that verifies an image, runs staging checks and then promotes it. Keep that sequence. Change the verification step for the selected prerelease, retain its output in the job record and confirm that failure still prevents promotion. Do not add a fallback that accepts the image whenever the new command fails.

For a useful negative check, run the verification step in an isolated test job with an intentionally incorrect expected identity. That job should fail before deployment. Then restore the documented identity and investigate any remaining failure rather than widening the accepted signer. This is a proposed acceptance check, not an executed test result.

Finally, record the exact image digest approved by your pipeline and keep that artifact consistent through staging and promotion. Otherwise, a successful check and the artifact eventually deployed can become two separate events in your process.

The useful outcome is a narrowly explained trust-policy change with a working failure path. A permanently disabled gate is not a repaired build.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help technical teams turn release changes and operational lessons into clear developer content without making them sound like marketing.

Book a 30-minute call