PARAS MADAN
All writing

Latest in AI

Together AI Prompt Storage: Check ZDR Before Changing Production

Check the setting that governs your Together AI API keys, plan for passthrough models to stop, and separate future retention from past exposure.

If your application uses a Together AI organization key, turning prompt storage off in your personal profile won't change what happens to its requests. The organization setting governs that traffic. Start by matching each production key to its organization, then verify the setting there.

In a November 21 disclosure, Together said an unauthorized party downloaded stored samples from a subset of its June and July serverless inference prompts. It also said zero data retention, or ZDR, is now the default for new accounts. The older setup docs still describe prompt storage as on by default. The notice does not say existing organizations were switched automatically, so check the value instead of inferring it from either page.

Find the setting that applies to your requests

In Together's organization privacy settings, an organization admin can go to Organization Settings → Privacy → Prompt storage. Turning that setting off enables ZDR for the organization. A key from that organization follows its setting even if the person who created the key chose something different in their own profile. The docs say organizations on the Limited tier need a payment method to change these settings.

Check Model training separately. An organization can decline training and still keep prompt storage on. If you have separate staging and production organizations, inspect both. A setting saved on the one you use for experiments doesn't tell you anything about the one that bills real customer traffic.

Check which models will stop working

The ZDR guide says enabling it also disables passthrough models. Together forwards those requests to an upstream provider, and the feature requires prompt storage. Models hosted on Together's own infrastructure can be authored by another company without being passthrough. The model name alone doesn't settle the question.

Make a short inventory of your primary and fallback model routes before switching. Check the full list in deployment settings, not just the one that shows up most in your billing chart. A fallback can sit unused for weeks, then fail when your usual model is busy. If you need to replace one, run synthetic requests through the substitute before moving production traffic. Check the operations you actually rely on, such as streaming, structured output or a specific image input. A successful text reply doesn't cover those paths.

Treat the June and July scope as a separate review

Together named MiniMax M3, GLM-5.2 and Kimi K2.6, and said the samples were primarily from June 24 and July 4, 8 and 14. The investigation is ongoing; the dates and model names are a starting point for review, not a list of affected requests.

Use existing request metadata, if you have it, to identify the services that called those models and what kinds of data they could have sent. You can usually answer that without copying old prompts into a ticket. If you identify a credential that could have been sent in an affected prompt, follow your incident process for replacing it and checking use. Don't paste the credential or prompt into a vendor support request.

Know what the switch does not remove

The Together privacy policy says ZDR applies from the moment you enable it. It does not change the treatment of data sent before that. Use the vendor's documented deletion process for older saved content if appropriate, with your incident response team coordinating what records need to be preserved.

The ZDR guide distinguishes request content from usage metadata such as model IDs and token counts. It also says explicitly uploaded batch and fine-tuning files remain stored until deleted. Your own gateway logs, tracing tools and application storage are separate again. If your app logs full request bodies, changing Together's setting doesn't turn that off.

Record the organization, date of the settings change and the replacement models you tested. That gives the next engineer something better than a note saying “ZDR is on.”

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams turn operational changes into technical writing developers can use.

Book a 30-minute call