How to Estimate LLM API Costs: A Token, Cache and Retry Worksheet

A model's price per million tokens is only one input to an API budget. A support assistant might make several model calls to finish one ticket, reuse some context, search a knowledge base and retry an interrupted request. Each of those choices changes the cost per completed ticket.

I would build the budget around a unit of useful work, such as one resolved ticket, one document extraction or one accepted code change, because that is the outcome the product actually needs to pay for. Estimate the calls and billable usage needed to complete that unit, then multiply by your expected volume.

The worksheet below uses hypothetical rates and a synthetic workload. It is a calculation example, not a provider quote or an API benchmark. The documentation was checked on November 6, 2026.

Separate the bill into token categories

Record these fields for each model and processing tier:

Category	What belongs here
Ordinary input	Input charged at the uncached rate, excluding tokens already assigned to a cache-read or cache-write bucket
Cache reads	Reused input charged at the cached-input rate
Cache writes	Input charged at the cache-write rate, split by lifetime when prices differ
Output	The provider's billable output count
Tool charges	Separately priced searches, execution sessions or other tool units
Other costs	Storage, external services and infrastructure that your workflow actually uses

Copy rates from the provider's current pricing page, recording the model ID, endpoint, processing tier, context-length band and currency beside them so the estimate can be checked later. The OpenAI pricing page illustrates why this matters: token rates vary by processing mode, and tools can add usage or storage charges. Cloud-hosted versions may have a different biller and rate card.

Don't estimate billable output by counting the words displayed to the user. OpenAI's reasoning guide explains that reasoning tokens are billed as output. A response can consume tokens before it produces a visible answer. If the usage total already includes those tokens, adding its reasoning subtotal again double-counts them.

The cost formula

Use prices in dollars per million tokens throughout:

model_cost = (
    ordinary_input_tokens * input_rate
  + cached_read_tokens * read_rate
  + short_write_tokens * short_write_rate
  + long_write_tokens * long_write_rate
  + output_tokens * output_rate
) / 1_000_000

workflow_cost = sum(model_cost for every billable call)
              + tool_fees + storage + other_services

cost_per_completed_task = workflow_cost / completed_tasks

The token buckets must not overlap. Where a rate card gives the total price for a cache write, don't charge the same tokens at ordinary input price as well.

Usage schemas differ. In the current OpenAI Responses API, for models supporting the newer caching fields, ordinary input is input_tokens - cached_tokens - cache_write_tokens. The prompt-caching documentation also splits writes by lifetime. Use those components instead of charging both the components and their parent total. Check another provider's schema before applying this subtraction.

Worked example: 10,000 support tasks

Suppose an assistant completes 10,000 tasks a month. For this example, each task needs one successful model call with 3,000 input tokens and 500 billable output tokens.

Of the input, 2,000 tokens are a reusable prefix and 1,000 are task-specific. Assume 8,000 successful calls read the full prefix from cache and 2,000 write it at the short-lifetime rate. This is an assumed reuse pattern, not a promised hit rate. There are no long-lifetime writes in this scenario.

Use this deliberately hypothetical rate card:

Token category	Example USD per million tokens
Ordinary input	$2.00
Cached read	$0.20
Short-lifetime write	$2.50
Long-lifetime write	$5.00
Output	$10.00

The successful calls produce the following monthly worksheet. Each token cost is tokens * rate / 1000000.

Line item	Calculation	Monthly cost
Ordinary input	10,000 calls × 1,000 tokens × $2 / 1M	$20.00
Cached reads	8,000 calls × 2,000 tokens × $0.20 / 1M	$3.20
Short writes	2,000 calls × 2,000 tokens × $2.50 / 1M	$10.00
Long writes	0 tokens	$0.00
Output	10,000 calls × 500 tokens × $10 / 1M	$50.00
Model subtotal	Sum of the five token rows	$83.20
Hosted tool fees	1,000 calls × hypothetical $0.01 per call	$10.00
Successful-work subtotal	Model subtotal + tool fees	$93.20

The input categories reconcile to 30 million tokens: 10 million ordinary, 16 million read from cache and 4 million written. The cached fraction of all input tokens is about 53.3%, even though 80% of successful requests have a cache hit. Those percentages measure different things.

Without caching, the same successful calls would cost $60 for input and $50 for output, plus $10 in tool fees: $120.00. Under these assumptions, caching saves $26.80 across the workload. It does not discount output or the separate tool fee.

Add retries from usage, not from error counts

For the same hypothetical month, suppose the reconciled usage records for additional processed attempts show:

Retry usage category	Billable tokens	Cost at the example rates
Ordinary input	50,000	$0.10
Cached reads	300,000	$0.06
Short writes	60,000	$0.15
Output	20,000	$0.20
Total	430,000	$0.51

These are synthetic usage totals, not a failure percentage multiplied by the price of a normal call. Assume there are no extra tool fees on these attempts. The combined estimate is $93.71, or $0.009371 per completed task, before storage, infrastructure, taxes and any other excluded charges.

Keep a request ledger with the logical task ID, attempt number, provider request ID, model, usage, tool charges and outcome. Record SDK retries too. This lets you count an attempt once while grouping all attempts under the task they served.

An error without returned usage is not proof of a zero-dollar charge. Mark its cost as unresolved and reconcile it against provider usage or billing records. A timeout also does not prove the provider stopped processing. Don't assign every failed attempt the full normal-call price, or silently drop attempts whose billing outcome is unknown.

Copyable calculator

This Python example reproduces both subtotals. It runs locally without an API key or network calls. Change the rates and usage totals to match your own records.

from decimal import Decimal

# Hypothetical USD rates per million tokens.
rates = {
    "input": Decimal("2.00"),
    "read": Decimal("0.20"),
    "short_write": Decimal("2.50"),
    "long_write": Decimal("5.00"),
    "output": Decimal("10.00"),
}

successful = {
    "input": 10_000_000,
    "read": 16_000_000,
    "short_write": 4_000_000,
    "long_write": 0,
    "output": 5_000_000,
}
retries = {
    "input": 50_000,
    "read": 300_000,
    "short_write": 60_000,
    "long_write": 0,
    "output": 20_000,
}

def token_cost(usage):
    if set(usage) != set(rates):
        raise ValueError("Usage and rate categories must match")
    if any(type(n) is not int or n < 0 for n in usage.values()):
        raise ValueError("Token counts must be non-negative integers")
    return sum(Decimal(n) * rates[k] for k, n in usage.items()) / Decimal("1000000")

tool_fees = Decimal("1000") * Decimal("0.01")
total = token_cost(successful) + token_cost(retries) + tool_fees
print(f"Successful model calls: ${token_cost(successful):.2f}")
print(f"Retry token usage: ${token_cost(retries):.2f}")
print(f"Monthly estimate: ${total:.2f}")
print(f"Per completed task: ${total / Decimal('10000'):.6f}")

Expected output:

Successful model calls: $83.20
Retry token usage: $0.51
Monthly estimate: $93.71
Per completed task: $0.009371

Check whether the assumptions survive real traffic

Cache savings depend on how much eligible content is reused before another write is needed. A longer cache lifetime can cost more to write. With the example rates, a 2,000-token prefix costs $0.005 for a short write or $0.010 for a long write; a read costs $0.0004. Choose the lifetime using the gaps between requests and the frequency of prefix changes.

Avoid treating a single average as a spending limit. For this worksheet, two changes have a larger effect than the retry total:

Changed assumption	Revised monthly estimate
Baseline	$93.71
Successful-call output doubles to 1,000 tokens; everything else stays fixed	$143.71
Every successful call uses the separately billed tool; everything else stays fixed	$183.71
Both changes occur	$233.71

These are sensitivity calculations, not forecasts. For an actual application, sample short and long conversations separately. Include retrieval results, tool definitions and prior turns in the input you measure. Track extra agent steps and rejected answers as part of the workload, even when the user eventually receives one short response.

A cheaper model can still be more expensive per accepted result if it requires more attempts or manual correction. Compare candidates on the same acceptance criteria and record review time separately from the API bill.

Apply batch pricing only to compatible work

OpenAI's Batch API offers a 50% discount compared with synchronous processing for supported work, with a 24-hour completion window. It suits an overnight extraction job more readily than an interactive support conversation.

Before adding a batch discount, check the model, endpoint, request parameters and tool support. Use the actual batch rate for each applicable token category. Don't halve a total that also contains storage or third-party fees. Check whether caching and batch pricing combine for that provider, and don't assume concurrent batch requests will produce the same cache reuse as sequential traffic.

Keep completed and unfinished requests separate when reconciling a batch. OpenAI documents that completed requests in an expired batch remain billable; retrying the whole input file would unnecessarily repeat completed work.

Keep subscription benefits separate from gross API spend

ChatGPT subscriptions and API usage have different meters. However, saying that a ChatGPT subscription can never include API credits is no longer accurate: OpenAI documents API credits for eligible ChatGPT Pro and Business subscribers, with activation, eligibility and expiry conditions.

Calculate gross API usage before applying the credits actually available to the relevant organization and billing period, and keep the subscription fee as a separate expense. A temporary credit balance can reduce cash paid this month without changing the cost of running the workload after the benefit ends.

Before using the estimate in a product budget, replace every hypothetical rate, reconcile a representative usage sample and list the excluded costs. Keep both the total spend and cost per completed task visible. That makes it easier to tell whether a rising bill comes from more customers, longer answers or additional work needed to finish the same task.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders turn product details and engineering decisions into clear technical writing and stories developers can use.

Book a 30-minute call