AI API Stream Stops Mid-Response: Detect Completion Before Retrying

When an AI answer stops halfway through a sentence, the tempting fix is to reconnect and send the prompt again. That may produce a readable answer, but it can also repeat work, hide a token limit, or leave your application displaying a fragment as though the job finished.

My recommendation is to make completion an explicit state in your application. Text arriving on screen should remain provisional until you have both the provider's completion evidence and whatever validation your product requires. A connection closing, a spinner disappearing or a paragraph ending neatly should never be enough to mark a task successful.

This guide focuses on OpenAI Responses API streams over HTTP and Claude Messages streams. The runnable example classifies decoded OpenAI events locally; it does not call either API, reconnect a socket or execute tools. Realtime, WebSocket steering and multi-agent streams need their own lifecycle handling.

First decide what actually stopped

An interrupted answer can come from several layers, and they need different fixes. Start with the evidence you have rather than treating every failure as a retryable network problem.

What you observed	What you can safely conclude	Next step
Connection closed without a final provider outcome	You lost confirmation of completion	Preserve the partial result and investigate the existing attempt
Provider returned an incomplete outcome	Generation stopped with a stated limitation	Inspect the reason before changing limits or requesting continuation
Provider returned a failure or stream error	The attempt needs error handling	Record the error and apply a bounded policy for that cause
Provider confirmed completion, but your JSON or business checks failed	Transport completion was insufficient	Validate or repair the result without calling it successful
User pressed Stop	The user chose to stop waiting	Keep that intent separate from an accidental disconnect

Keep your own task identifier alongside each provider attempt. A useful record contains the response or message ID, the model, timestamps, the last observed lifecycle event, the terminal outcome, and whether any external action has already happened. Avoid turning observability into a second store of sensitive prompts; redact payloads and follow your retention policy.

OpenAI: wait for the response outcome, not the last text fragment

The OpenAI streaming guide exposes typed events such as response.output_text.delta, response.completed and error. Keep those lifecycle events in your handler instead of reducing the stream to text alone.

The streaming event reference distinguishes response.completed, response.failed and response.incomplete, each carrying a response object. response.output_text.done finalizes a text part, while response.output_item.done finalizes an item; neither is the whole-response completion event. An incomplete response carries incomplete_details, which can explain a limit such as max_output_tokens.

For your application, even a completed response is only the next checkpoint. Inspect the final output for refusals, tool requests and the content you actually need, then apply schema and business validation. A polished sentence is not proof that a booking was made, a database write succeeded or a calculation was correct.

A small completion classifier you can test offline

Save the following as classify-stream.mjs and run it with node classify-stream.mjs. It accepts already decoded event objects from one response attempt, so use an SDK or proper SSE parser before this layer. The synthetic fixtures show only fields this classifier reads. It deliberately does not parse raw network chunks or concatenate multiple output items.

import assert from "node:assert/strict";

export function classifyAttempt(events, { userStopped = false } = {}) {
  const terminalTypes = {
    "response.completed": "completed",
    "response.failed": "failed",
    "response.incomplete": "incomplete",
  };
  let responseId = null;
  let terminal = null;
  let streamError = null;

  for (const event of events) {
    if (!event || typeof event.type !== "string") {
      return { state: "invalid_event", responseId };
    }
    if (event.type === "keepalive") continue;

    const id = event.response?.id;
    if (id !== undefined) {
      if (typeof id !== "string" || !id ||
          (responseId !== null && id !== responseId)) {
        return { state: "mixed_or_invalid_response", responseId };
      }
      responseId = id;
    }

    if (event.type === "error") streamError = event;
    if (!Object.hasOwn(terminalTypes, event.type)) continue;

    const expected = terminalTypes[event.type];
    if (!id || event.response.status !== expected) {
      return { state: "invalid_terminal", responseId };
    }
    if (terminal && terminal.state !== expected) {
      return { state: "conflicting_terminal", responseId };
    }
    terminal = { state: expected, response: event.response };
  }

  // A contradictory error deserves review, not an optimistic success label.
  if (streamError) {
    return { state: "stream_error", responseId, error: streamError };
  }
  if (terminal) return { ...terminal, responseId };
  return { state: userStopped ? "user_stopped" : "interrupted", responseId };
}

const created = {
  type: "response.created",
  response: { id: "resp_demo", status: "in_progress" },
};
const textDone = {
  type: "response.output_text.done",
  text: "This looks complete.",
};
const completed = {
  type: "response.completed",
  response: { id: "resp_demo", status: "completed", output: [] },
};

assert.equal(classifyAttempt([created, textDone]).state, "interrupted");
assert.equal(classifyAttempt([created, textDone, completed]).state, "completed");
assert.equal(classifyAttempt([created], { userStopped: true }).state, "user_stopped");
assert.equal(classifyAttempt([
  created, { type: "error", code: "server_error", message: "Example" },
]).state, "stream_error");
console.log("Four offline completion checks passed.");

The empty output in the completed fixture is intentional: it demonstrates why the classifier reports a provider lifecycle state rather than an acceptable answer. Your next layer must decide whether the result satisfies the task.

If your stream reader throws, preserve the events processed before the exception and classify that attempt. If it simply reaches the end without a terminal event, take the same interrupted path. Unknown event types do not magically imply success, and the classifier leaves them alone rather than crashing on every future event addition.

This is a diagnostic example, not a production event store. For a live application, process events incrementally, bound memory use, persist state durably, and separate each retry into a new attempt. The local fixture tests exercise classification only; they do not verify network behavior, SDK retries or provider-side execution.

Recover an OpenAI background response without creating another job

OpenAI's background mode guide supports reconnecting to a response created with both background=true and stream=true. Keep its response ID and the last stored event's sequence_number, then retrieve the stream using starting_after; exclude keepalive events because their counters are not stored-event resume positions.

That is a specific recovery route, not a promise that every ordinary stream can be replayed. Background requests can use store=false, but data is temporarily stored to support execution and polling. The current guide says ZDR background requests use that setting with roughly ten minutes of temporary disk storage, so do not describe background mode as either blanket ZDR-incompatible or storage-free.

In your own storage, commit the processed event and its checkpoint together. Saving the cursor first creates a nasty failure: after a crash, you can resume beyond text or state that your application never saved. Make rendering safe to repeat as well, so a recovered event cannot append the same paragraph twice.

If the existing response cannot be recovered, record that limit before creating another attempt. A newly generated continuation may change wording or repeat a boundary paragraph, so keep the original fragment and the continuation identifiable until you have checked the combined result.

Claude uses a different completion contract

Claude's streaming documentation ends a normal message stream with message_stop and allows errors inside the stream. Its SDK can accumulate the final message, which is preferable to writing a fragile text-only collector when you do not need custom event handling.

Then inspect the stop reason. end_turn signals a natural finish, max_tokens or model_context_window_exceeded signals truncation, and tool_use requires the tool workflow to continue. A final message can therefore be valid without containing the final answer your user expected.

For interrupted text on Claude 4.6 and later, the documented continuation approach puts the captured fragment and a continuation instruction in a new user message. Do not copy the older assistant-prefill recovery pattern intended for 4.5 and earlier; partial tool-use and thinking blocks are not recoverable through that text-continuation method.

Keep provider-specific adapters rather than converting every event containing the word “stop” or “done” into one success flag. A small amount of explicit branching is easier to audit than a universal handler that quietly erases meaningful differences.

Put a limit on recovery, especially when tools have side effects

Before replaying a request, check what happened outside the text stream. If an agent may have sent a message, submitted a form or changed a record, reconcile that action using your application's durable task and operation IDs. Reconnecting the display and repeating the action are different decisions, and a transport failure does not prove that an external service did nothing.

Set a maximum number of recovery attempts, a total elapsed-time limit and a cost ceiling. When one is reached, preserve the partial result with a clear status and offer a deliberate next step rather than spinning forever. For user cancellation, stop automatically starting fresh work; if a terminal outcome arrives during the cancellation race, record both the actual outcome and the user's intent.

Before shipping, test a dropped connection before the first event, after some text, after a text-part completion, and after the final response event. Add stream errors, incomplete outcomes, mixed response IDs, malformed terminal payloads and explicit user cancellation. Also test a completed response whose output fails your application validation, because that is the case a transport-only test suite will miss.

I'm Paras, a technical writer working with AI and developer-tool teams.

I help technical teams turn API behavior and engineering decisions into clear developer guides and product stories.

Book a 30-minute call