Latest in AI
NVIDIA Snapshot: GPU Cold-Start Requirements and Restore Troubleshooting
Checkpointing can skip model initialization, but only when the workload, storage and destination GPU satisfy the restore contract.
NVIDIA Snapshot restores initialized GPU workloads instead of loading weights and rebuilding runtime state for every start. NVIDIA's November 10 announcement makes it attractive for inference autoscaling, but the useful question is whether your particular workload can resume safely on another node. A fast restore benchmark does not answer that.
The right evaluation starts with compatibility and application readiness, then measures request-serving latency. These are documentation-based checks, not results from an independent deployment test.
What Snapshot removes from a cold start
NVIDIA's benchmark reports restore times between 3.6 and 42.7 seconds across its tested configurations. The measurements exclude container setup and image pulls. Provisioning a GPU node is also outside the restore operation, so those numbers should not become an autoscaling service-level promise.
For example, NVIDIA reports that its single-GPU Qwen3-14B vLLM configuration took 122.76 seconds to start cold and 5.72 seconds to restore. Creating that checkpoint took 19.98 seconds and produced a 37.71 GB artifact. These are vendor measurements on the documented test stack, not a prediction for a different storage system.
That distinction changes the economics. A checkpoint has an upfront creation and storage cost that repeated restores can amortize. If every deployment changes the model, image or initialization settings, the team needs to measure how often a reusable checkpoint actually survives those changes.
Check the GPU, driver and storage requirements first
The current prerequisites specify Linux 5.13 or later, supported x86-64 or Arm64 hosts, Kubernetes with containerd or CRI-O, GPU Operator 26.3 or later, and an NVIDIA kernel-mode driver from the 580 series or newer. MIG must be disabled.
Do not confuse that minimum driver requirement with freedom to restore across arbitrary machines. The destination needs a compatible GPU and driver environment; NVIDIA's launch guidance specifies the same GPU model and driver version. CPU compatibility matters too, because captured processes can retain compiled instructions that the target CPU does not support.
The storage-backed setup uses a ReadWriteMany volume. An S3 backend is documented, but requires matched components from a release newer than v0.2.0. Confirm the installed chart, agent and PageBroker versions rather than treating a current main-branch feature as available in an older release.
This is privileged infrastructure: the agent requires host access through a privileged DaemonSet with host PID, IPC and networking. Evaluate it in a controlled cluster before proposing it as a small application-only change.
Make application readiness part of the checkpoint
The workload contract separates snapshot readiness from serving readiness. Before capture, stop generation requests from running. Warming up a real generation is recommended so that initialization work is already done, and releasing KV-cache allocations can reduce the checkpoint artifact.
After restore, the process continues from captured state. Its entrypoint does not simply run again. Application logic must wait for the restore-complete signal, restore the GPU memory it needs, and expose serving readiness only after the API socket is listening.
This deserves a deliberate failure test. Restore a checkpoint while the load balancer is polling readiness, then check whether any request reaches the server before the engine is usable. A pod that is running is not enough evidence that an inference endpoint is ready.
Diagnose common restore failures
A CUDA invalid-device error in a multi-GPU workload can indicate the special requirements around NCCL NVLS multicast groups. The prerequisites call for user-mode CUDA driver 615 or newer for that case. Ordinary single-node multi-GPU support should not be mistaken for support for every multi-GPU configuration, and multi-node snapshotting is not currently supported.
A process that exits with an illegal-instruction error points toward CPU compatibility. A process that restores but cannot reopen a file points toward the captured filesystem and file-handle contract. A healthy-looking container that never serves requests calls for inspection of the restore and readiness handshake before another infrastructure restart.
Record four timings separately: node availability, container setup, snapshot restore and first successful request. The number that matters to users is the last one. Snapshot is useful when model initialization dominates that path; it cannot eliminate the parts of the path that it does not own.
I’m Paras. I work at the intersection of AI, engineering and distribution.
I help technical teams turn infrastructure work and product updates into clear stories that developers can use and share.