PARAS MADAN
All writing

Latest in AI

Qwen3 TTS on Together AI: When to Use HTTP or WebSocket Streaming

Choose a streaming API for Together's Qwen3 TTS, decode raw audio correctly, and measure the time to first audible speech in your app.

If your app already knows what it wants to say, send the full line to Qwen3 TTS in one HTTP request and stream the audio back. If a language model is still writing the line, use a WebSocket so synthesis can start before the entire reply is ready. These solve two different waits, and the second is worth the extra code only if text generation is on the critical path.

Together AI added Qwen3 TTS on November 20. Use the model ID Qwen/Qwen3-TTS-12Hz-1.7B. Together offers nine preset voices across ten supported languages, with style instructions that control delivery. This is a new hosted route for an existing model.

Start with a line you already know

Imagine an order-status assistant. If the lookup has already returned the shipment date and your app fills a fixed template, the HTTP streaming API is enough. Send the complete input with stream=true, response_format="raw" and response_encoding="pcm_s16le". Start with voice="aiden" and language="en" for the English test. With the Python SDK, put an optional instructions string in extra_body.

The response is a stream of server-sent events, not a WAV file. Decode the delta field on each conversation.item.audio_output.delta event from base64. For the launch example's settings, feed those bytes into a mono signed 16-bit little-endian PCM player at 24 kHz. If you save a fixture instead, mark it as raw PCM and pass the sample rate and channel count to the player. A .wav extension won't add a header or make the file playable.

Switch when the reply is still being written

Use the WebSocket API at wss://api.together.ai/v1/audio/speech/websocket from your backend and authenticate with the Authorization header. The raw format here is named pcm, rather than HTTP's raw. Keep the API key off the browser and out of URLs.

Read audio events while you send text with input_text_buffer.append. The default mode buffers until a sentence is complete or a length threshold is met. Send input_text_buffer.commit at the end so a final fragment without punctuation doesn't sit in the buffer. Handle failure events and don't assume the first audio item to finish means your whole reply has played.

Try a reply with two sentences: a short status, then a longer explanation. Feed them at different times and check that you hear the first before the second is ready. If you get no benefit, look upstream: your app may be holding all text until the model finishes, or the first sentence may be so long that synthesis starts late anyway.

Treat interruption as a playback problem too

The socket supports clearing buffered text, but that does not retract text already being synthesized. A phone caller can interrupt after your player has already queued half a sentence. On interruption, stop that local queue and reject late audio from the interrupted turn. Clearing the provider buffer alone won't stop already scheduled audio from playing.

Test a pause in the model stream, an interruption mid-sentence, and a socket disconnect halfway through a turn. A retry before the caller hears anything is different from replaying an order update they already heard.

Measure the first sound the caller actually hears

Together reports first audio in as little as 97 ms. That isn't a percentile or an end-to-end voice latency promise. Log when usable text is ready, when your first decodable chunk arrives, and when the player actually starts. Measure both median and p95 with the network and player your users have. If the gap between the last two timestamps is large, changing the TTS API may do less than fixing buffering.

Use real phrases for the listening check: order numbers, addresses, local date formats and names. Set the language of the input rather than expecting the endpoint to translate it. If your UI needs word-level highlighting, don't assume the general streaming API's timestamp support applies here: Together says this Qwen3 model doesn't support word alignment.

Budget by characters, not audio minutes

The serverless model page lists $20 per million characters. At that rate, 10,000 replies of 150 characters each use 1.5 million characters, or $30 for synthesis alone. Compare that with actual generated text volume, including retries, and budget model generation and telephony separately.

I'd start with HTTP for templated messages and test WebSocket only on replies that users currently wait to hear. That gives you a baseline and tells you whether running the two streams concurrently is worth maintaining.

I'm Paras. I work at the intersection of AI, engineering and distribution.

I help AI teams turn speech APIs and integration details into technical writing developers can use.

Book a 30-minute call