vLLM 0.33 Upgrade Guide: CUDA 13, Decisions API and A100 Checks

vLLM released version 0.33.0 on November 2 with PyTorch 2.14, CUDA 13.0 release artifacts and an OpenAI-compatible Decisions API. For teams running their own inference servers, the immediate question is which deployment assumptions need changing before the new features reach production.

The release notes include support for models such as Reflection Beam-1.0, Muse Spark Open and Step-5 Preview. They also remove older execution paths. The useful upgrade plan starts with the runtime, then checks application behavior and performance separately.

CUDA 12.9 artifacts are gone

CUDA 13.0 was already the default in vLLM 0.32. The change in 0.33 is that the alternative CUDA 12.9 wheels and Docker images are no longer published. Deployments using tags ending in -cu129 or -cu129-ubuntu2404 need a different artifact strategy.

The release-pipeline change removes those builds rather than updating them alongside CUDA 13.0. Changing only the vLLM version inside an existing image name can therefore leave a deployment pointing at an artifact that does not exist.

Check the GPU driver, container runtime and custom extensions against the chosen image before rolling it out. A working Python import is a useful first check, but the real test is loading the production model and running its actual kernels.

AMD deployments have a separate default change: the ROCm image and wheels move to ROCm 10.0, with ROCm 7.2 retained as a separate variant. Teams deliberately staying on the older stack should pin that variant explicitly rather than inherit the new default.

What the Decisions API adds

The versioned structured-decisions documentation describes POST /v1/decisions, a text-only implementation of the OpenAI request format. It supports predicate, choice and score questions on generation models with a registered read strategy. Unsupported models return HTTP 501.

This sits alongside /v1/systemone, which retains its existing request format. An application already using System One does not need to rename its endpoint simply because another format is available.

The backend reads probabilities for answer labels. A support workflow could ask whether an item arrived damaged and which department should receive the ticket, without asking for a paragraph that another service must parse.

There are important boundaries. The implementation allows up to 64 questions, 26 choices and ten score levels. It excludes images, tools, streaming and per-question refusal scoring. Its confidence values are not calibrated to OpenAI's models.

An API-compatible request does not establish equivalent decision quality. If a router sends uncertain cases to a human, validate that threshold against labeled examples from the actual workflow. Reusing another model's threshold can change who receives the difficult cases even when the response schema stays familiar.

Check the A100 regression and the exact build

An early performance report identified extra CUDA-graph replay overhead in a mixture-of-experts routing kernel on pre-Hopper GPUs. The report connects it to a launch attribute added in 0.33.0, and distinguishes captured graph replay from eager execution, where it did not observe the same effect.

The size of the application-level impact varied. In the author's public-model A100 tests, the proposed fix improved batch-one latency by roughly 0.7% for OLMoE and 1.2% for a quantized Qwen3 model. Batch-16 results were within noise. Larger gains reported for a private checkpoint should not be treated as a general vLLM benchmark.

The fix merged into main on November 2. It restricts the launch attribute to SM90 and newer GPUs. That is a meaningful development, but a merged fix does not retroactively change a previously built 0.33.0 image.

Before relying on it, check whether the exact artifact being deployed contains the fix. For A100 workloads, compare the same model, quantization, concurrency and graph settings against the existing production build. This is a targeted regression check, not evidence that every 0.33 deployment is slower.

Upgrade the workload before adopting new defaults

The release also removes Medusa, the MLP speculator and the legacy Ray compiled-graph executor. Audit launch commands and saved configurations for those dependencies before scheduling a rollout.

Keep a rollback image and measure first-token latency, sustained decode speed and output quality with the existing configuration. Then introduce the new endpoint or a different execution path independently.

The practical opportunity is to make model-serving upgrades easier to evaluate. When runtime compatibility, API behavior and hardware-specific performance have separate checks, a useful new capability can reach production without making every unrelated change part of the same bet.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn infrastructure releases and engineering changes into clear launch stories developers can understand and use.

Book a 30-minute call.