PARAS MADAN
All writing

Latest in AI

GKE Agent Session Parking: Size the Worker Pool Before Claiming Savings

Suspending idle agents can reduce compute costs, but the result depends on burst traffic, restart behaviour and the capacity you keep warm.

Google's November 16 GKE guide models a 40% node-bill saving from parking idle coding-agent sessions. That result assumes 100 agents, two prompts per hour, a two-minute idle timeout and an after-hours shutdown schedule. It is a useful sizing example, not a discount every agent platform can expect.

For a team moving agents from laptops to a shared cluster, the first question is how many sessions need compute at the same time. An open conversation can sit untouched while its allocated machine continues adding to the bill.

Start with the busiest hour

In Google's cost model, the parked configuration uses 15 worker nodes plus an always-on control-plane node, compared with 34 nodes for one Pod per session. At four prompts per hour, the modeled saving drops to 4%. The comparison uses Google Cloud list prices in us-central1.

These figures depend on workload assumptions, including independent session activity. A team that starts agents together after a morning meeting may behave differently. Collect prompt times, turn durations and worker occupancy, then examine the busiest periods instead of sizing from the daily average.

For example, imagine 100 open sessions but only 20 usually executing. That does not establish that capacity for 20 is sufficient. Builds, tests, resume operations and synchronized prompts can overlap. Treat that count as a starting observation and measure the tail before choosing headroom.

Compare the same CPU guarantee

Agent Substrate's versioned API guide explains that workers book actor resource limits against their remaining capacity. Runtime overhead also consumes worker capacity. An actor's advertised CPU and memory limits therefore matter to placement, not just to what happens after it starts.

A cheaper comparison can hide a weaker guarantee. If the existing Pods request a fraction of their CPU limit, they already share capacity more aggressively. Comparing those Pods with actors that reserve their full limit would mix a runtime change with an allocation-policy change.

Write down the same resource assumptions on both sides: CPU available during contention, memory, operating hours and acceptable task latency. Include control-plane, storage and transfer costs separately. A node-only saving does not establish the same percentage reduction in the entire platform bill.

Give idle suspension an owner

The project's inactivity issue remains open. Agent Substrate requires an explicit suspend call; merely deploying it does not create an idle-timeout policy. The November 16 discussion describes an actor-callable suspend API as the direction of work, rather than a completed automatic idle detector.

Choose which component decides that work has finished: the agent harness, a front-end service or another controller. Low CPU alone is a poor signal for an agent waiting on a network response. A useful policy must distinguish waiting for a person from waiting for a tool that is still doing work.

Test the boundary with a disposable task. A long-running command should not be parked just because no new prompt arrived. After a completed turn, the policy should eventually release capacity. Record both outcomes before treating an idle-timeout setting as a working control.

Verify what survives a resume

The AX runner contract says its current suspend-and-resume path preserves the workspace volume but starts a fresh container and process tree. A resumed task has the same files, not necessarily the same in-memory application state.

That changes the acceptance test. Save a synthetic file, suspend the task, resume it and verify the file contents. Separately check whether the agent can reconstruct its working state and continue without repeating a consequential action. Persistence and correct continuation are different requirements.

The runner documentation also warns against re-running workspace setup on every resume. A fresh clone over restored files can destroy state. Make initialization distinguish a new workspace from a resumed one, and verify that distinction with an intentionally modified file.

Treat a full pool as a user-visible failure

The request-parking documentation describes bounded retry when an actor cannot obtain capacity. Its default retry budget is five seconds, after which unresolved capacity problems can surface as HTTP 503. An already committed restore can finish later, so five seconds is not a hard end-to-end response deadline.

Measure resume latency and capacity errors alongside cost. A smaller warm pool that regularly delays developers may be cheaper on an invoice and worse for the team. Include a burst test and a recovery test, with explicit limits for waiting and failed prompts.

This is a documentation-based planning guide, not a cluster benchmark. Parking is worth evaluating when sessions spend substantial time idle and the application can resume correctly. Keep one-Pod-per-session as the baseline until the same workload demonstrates a better total result.

I’m Paras. I work at the intersection of AI, engineering and distribution.

I help infrastructure teams explain technical trade-offs through practical guides developers can use.

Book a 30-minute call