# Cloud Run Fractional GPUs: When 48 GB of Blackwell VRAM Is Enough

Size a fractional Cloud Run GPU service, check the preview limits and compare what you pay per completed inference request.

By Paras Madan | 2026-11-20


Cloud Run now lets you allocate half an NVIDIA RTX PRO 6000 Blackwell GPU. The interesting question is what you can serve from its 48 GB of video memory without giving up too much concurrency or losing the cost savings to additional instances.

Google lists fractional GPUs for services, jobs and worker pools in its [November 18 Cloud Run release notes](https://docs.cloud.google.com/run/docs/release-notes#November_18_2026). This guide focuses on an HTTP inference service, where the sizing decision depends on what happens across many requests, not just whether the model starts.

## Check the region and resource floor

The [Cloud Run GPU service guide](https://docs.cloud.google.com/run/docs/configuring/services/gpu) lists the 0.5 GPU preview in Iowa and the Netherlands. Full Blackwell GPUs also have other regions, including Delhi and Singapore, but those are not listed for the fractional option. If a workload has to stay in Delhi, the 48 GB option is not an in-region substitute.

Fractional Blackwell requires 10, 12, 14 or 16 vCPUs and at least 40 GiB of instance memory, with at most 4 GiB per CPU. The 48 GB of GPU memory is separate. A small request handler does not remove the need to provision that CPU and memory floor.

For comparison, full Blackwell provides 96 GB of VRAM and starts at 20 vCPUs and 80 GiB of instance memory. The L4 has 24 GB of VRAM and does not offer fractional allocation. Compare all three if your workload is small enough; the half Blackwell is not necessarily the smallest suitable instance.

## Prove that the model fits under load

Start with the exact checkpoint, precision, runtime and maximum input length you expect to use. A checkpoint that fits on disk says little about whether the runtime will fit once it allocates its cache and starts processing simultaneous requests.

For example, take an internal document summarizer with two traffic patterns: short single-document requests and a burst of longer documents every morning. Run the same prompts on the full and fractional configurations in the same region. Test the bursts separately rather than letting their failures disappear into a daily average.

Record model startup time, request latency at the tail and completed requests per minute. Raise concurrency one step at a time, and stop when you reach your latency budget or see memory-related failures. A smaller allocation that needs twice as many replicas may still be useful, but the trade-off should show up in your numbers.

## Check runtime compatibility and monitoring separately

Google's [known issues list](https://docs.cloud.google.com/run/docs/known-issues) says fractional GPUs do not support CUDA Virtual Memory Management or the Cloud Run Container GPU utilization metric. Those are two different constraints: one can prevent a runtime feature from working, and the other removes a familiar monitoring signal.

Smoke-test the exact container image you intend to promote, including initialization and the longest workload it is expected to handle. Do not assume every feature that worked on the full GPU is interchangeable.

For the trial, monitor successful requests, errors, queueing and tail latency. An empty GPU utilization chart on the fractional revision is not evidence that the accelerator is idle.

## Compare cost per completed request

Use the [Cloud Run pricing page](https://cloud.google.com/run/pricing) together with the GPU service guide to estimate the bill for the chosen region and redundancy setting. GPU services use instance-based billing, and a configured minimum instance still costs money when it is idle.

Compare the estimated cost of the whole service during the same test window, including CPU, instance memory and GPU time. Divide it by the number of successful requests that met your latency target. That gives you a useful comparison even if one shape uses more replicas or completes fewer requests.

Repeat the calculation for both an idle-heavy day and the busiest period you expect. An option that wins with bursty traffic can lose if your startup target requires keeping additional instances warm.

## Keep the trial reversible

Deploy a separate test service or use an isolated revision before shifting production traffic. For an existing service, the GPU guide documents `--gpu 0.5` with `--gpu-type nvidia-rtx-pro-6000`; check the region, resource settings and instance cap together. The allocation consumes 500 milliGPUs per instance, so check project quota before load testing replicas.

Write down the workload and latency threshold at which you would keep the larger GPU, and who can roll the change back. The fractional option earns its place when it serves the same real workload within budget, not just when it runs the same model.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams explain technical products through clear documentation, practical guides and developer-focused content.

[Book a 30-minute call](https://cal.com/paras-madan/30min).
