OCI Responses API Adds Context Compaction: How It Works and What to Preserve

Oracle added explicit and automatic context compaction to the OCI Responses API on October 27. The feature condenses accumulated conversation history for long-running agents, with support across models available through the API. Developers get more control over context growth, but compression can still lose details that a workflow needs.

What changed in the OCI Responses API

The release note describes two ways to compact history. An application can request it deliberately, or ask OCI to do it when a token threshold is reached. Both modes support on-demand inference and dedicated hosting, subject to the chosen model's availability in Enterprise AI regions.

This is an addition to an existing API, not a new model or a larger context window. It changes how an application carries information between requests. That distinction matters when an agent spends many turns collecting records, calling tools and revising an answer.

A larger context window postpones the point at which history becomes too large. Compaction gives the application another way to continue, but introduces a question that developers need to test: which earlier details remain usable afterward?

Explicit and automatic compaction

Oracle's API documentation describes the operational difference.

Mode	Request	History to carry forward
Explicit	POST /responses/compact	The complete returned output array, followed by the next user message
Automatic	POST /responses with context_management	With store=False, the latest compaction item and every later output item

If automatic compaction returns no compaction item, retain the existing history and append the new output. The items contain opaque encrypted context. Applications should preserve their fields and order rather than trying to rewrite them as an ordinary text summary.

Automatic compaction requires an explicit compact_threshold of at least 1,000 tokens. A sensible threshold also leaves space below the model's maximum context for its answer and subsequent turns.

Setup uses OCI credentials, even with the OpenAI SDK

The compaction tutorial uses the OpenAI Python SDK as a client for an OCI endpoint. Requests authenticate with OCI, not an OpenAI API key. An OCI Generative AI project OCID is required for every model and is passed through the client's project argument.

Oracle specifies Python 3.10 or later and openai>=2.45.0 for its examples. An on-demand request uses a supported model ID. Dedicated hosting instead uses the model endpoint's OCID, with the client pointing at the matching region.

These details are worth checking before debugging the prompt. A missing project, an incorrect regional endpoint or an unavailable model can prevent a valid compaction request from working.

Compaction has its own cost and failure modes

Explicit compaction is synchronous and does not support streaming, background execution or storage. Compaction itself consumes model tokens, and Oracle warns that summarization can lose information. A smaller next request therefore does not guarantee lower total token use or faster completion.

Consider an agent preparing a deployment change. Early in the conversation, a user rules out a particular region. Several tool calls later, that constraint might be less prominent than the latest logs. If the condensed history fails to retain it, the agent could produce a technically plausible plan for the wrong environment.

The useful test is whether the right constraints survive, not merely whether compaction occurred. Keep important approval state and operational constraints in application-managed records where appropriate, and check them before consequential tool actions. Conversation compression should not become the only place a safety-critical condition is stored.

Evaluate the completed workflow

Compare a representative task using full history, explicit compaction and automatic compaction. Record whether required facts survive, whether tool arguments stay correct and whether a reviewer has to reconstruct missing context. Include the compaction calls when measuring total cost and latency.

That produces a more useful decision than choosing the smallest possible threshold. The goal is a history-management policy that keeps the agent useful over a long task without quietly making its earlier commitments less reliable.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help technical founders explain APIs, agent infrastructure and engineering decisions in clear stories developers can use.

Book a 30-minute call.