Pick on replaceability, not latency. A speech-to-speech model is one component with nine plugin providers in LiveKit Agents; an STT-LLM-TTS pipeline is three components with 30, 21 and 38. OpenAI deprecated the legacy realtime family on 20 July 2026 for removal on 20 January 2027.
The question as people actually type it is: should I use a speech-to-speech model or stt plus llm plus tts. Everyone answers it on latency and naturalness, and both of those are settled and small. The criterion nobody prices is what happens the morning a model you depend on gets a shutdown date — and in the last sixty days gpt-realtime, gpt-audio and whisper-1 all got one, GPT-Live 1 reached general availability at $0.05 per minute, and every Google Gemini Live API model ID still carried a -preview suffix. This page answers the architecture question only; for the per-minute arithmetic read what an AI voice agent actually costs per minute, and for provider-by-provider schedules read model deprecation risk for AI agents.
The criteria that decide this, in the order they bind
Criteria first, because a comparison table is only useful once you know which row outvotes the others. These bind in order: a constraint higher in the list cancels everything below it.
- Control over the exact words spoken. Binary, not a trade-off. LiveKit states that realtime models offer no method to generate speech from a text script, that instructed output is not guaranteed to precisely follow any provided script, and that full-duplex models cannot speak a script word for word. If a disclosure or a quoted price must leave the agent verbatim, a pure speech-to-speech path is out at step one.
- Auditability and per-stage tracing. LiveKit rates the pipeline as a full text trail, a realtime model as limited, and a half-cascade as output text only. Realtime models produce no interim transcripts, and user transcriptions can arrive after the agent has already replied.
- Lifecycle exposure. The written notice you are owed, by vendor and by stage. This is the criterion this page exists for.
- Vendor count at each seam. How many alternatives sit at the boundary you would have to cross.
- End-to-end latency. LiveKit puts a streamed cascade at 300 to 600 ms and speech-to-speech at 200 to 300 ms.
- Interruption handling and prosody. Speech-to-speech hears tone; a transcript does not carry it.
- Published cost per minute. Real, but it moves least of anything here.
- Language and voice coverage. Deepgram Nova-3 alone lists over 60 language codes.
The comparison table, with values rather than adjectives
Every figure read on 14 September 2026 from the page of the organisation that owns it.
| Criterion | Speech-to-speech | STT-LLM-TTS pipeline | Half-cascade |
|---|---|---|---|
| End-to-end latency (LiveKit guide, 23 Mar 2026) | 200–300 ms | 300–600 ms streamed; 1,000–2,000 ms+ unstreamed | Moderate |
| Published rate, one named example | gpt-live-1 $0.05/min, backend billed separately; gemini-3.1-flash-live-preview $0.005/min in, $0.018/min out | gpt-transcribe $0.0045/min; Deepgram Nova-3 streaming $0.0077/min; Aura-2 $0.030 per 1,000 characters | Realtime rate plus the full TTS leg |
| Speaks an exact script | No (LiveKit: no say() equivalent) |
Yes | Yes |
| Interim transcripts during the turn | No; delayed, and can arrive after the reply | Yes | Delayed on input |
| Auditability | Limited | Full text trail at every stage | Output text only |
| Providers with a LiveKit Agents plugin | 9 realtime | 30 STT, 21 LLM, 38 TTS | 9 realtime and 38 TTS |
| Lifecycle stage of the leading models | gpt-live-1 GA; every Gemini Live API and native-audio model ID carries -preview |
gpt-transcribe and Deepgram Nova-3 both GA | Inherits the realtime side |
| Written notice before shutdown | OpenAI at least 6 months for GA, about 2 weeks for preview; Google at least 2 weeks for preview; AWS Bedrock 6 months or 45 days | Same policies, three independently dated components | Two clocks, not one or three |
The blast-radius table: what a shutdown date actually forces you to rebuild
This is the part the latency comparisons skip. A deprecation is not a config change if the deprecated thing has no boundary around it. The July 2026 event is the worked case: on 20 July 2026 OpenAI deprecated gpt-realtime, gpt-audio, gpt-4o-realtime, gpt-realtime-mini and their siblings for removal on 20 January 2027, then on 26 August 2026 deprecated whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe and gpt-4o-transcribe-diarize for removal on 26 February 2027.
| When one model gets a shutdown date | Speech-to-speech | STT-LLM-TTS pipeline |
|---|---|---|
| What you must replace | The whole conversational surface in one swap: comprehension, reasoning, turn-taking, voice | One of three components |
| What survives untouched | Tools, business logic, CRM writes, telephony | All of that, plus the prompt, the evals, the transcript corpus, the tuned endpointing and the other two components |
| Alternatives at that seam | 9 plugin providers | 30 (STT), 21 (LLM) or 38 (TTS) |
| What you must re-evaluate | Everything at once: voice, reasoning and timing moved together | The replaced stage, against transcripts you already hold |
| Number of independent clocks to track | 1 | 3, and they do not align: transcription to 26 Feb 2027, the GPT-5 and o3 snapshots to 11 Dec 2026, and no announced end for text to speech |
| Worst case if the replacement behaves differently | Rewrite the prompt and re-tune turn-taking, with no text trail to diff | Diff old and new transcripts on the changed stage |
Two corrections, both checked at source on 14 September 2026. The July deprecation is not an orphaning: every entry names a replacement, gpt-realtime to gpt-realtime-2.1 and gpt-audio to gpt-audio-1.5. And no text-to-speech model appears anywhere on the OpenAI deprecations page, upcoming or past. The pipeline’s three components therefore sit on three clocks, one with no announced end at all, and that asymmetry is the whole argument.
The seam is the insurance
The seam is the insurance: you can only swap out what you had a boundary around, so a pipeline’s extra latency is the premium you pay for replaceability. The useful part of that rule is that both sides are quantifiable, which turns a taste argument into arithmetic.
The premium, on LiveKit’s figures, is roughly 100 to 300 ms. The payout is the difference between replacing one component from 21 to 38 candidates and replacing the whole conversational surface from 9. Apply it as a threshold, not a preference:
| If this is true of your deployment | Then | Because |
|---|---|---|
| The exact spoken words must be scripted, approved or logged verbatim | Pipeline or half-cascade. No exceptions | Realtime output is not guaranteed to follow a script, per LiveKit |
| Your p50 end-to-end budget is under 300 ms | Speech-to-speech, and accept a 9-provider seam | A streamed cascade starts at 300 ms |
| Your budget is 300 to 600 ms | Pipeline. The seams are free here | Streaming overlaps the stages |
| Your planning horizon is longer than the longest notice you can find in writing | Keep at least the output seam | 6 months is the best GA commitment on the three policy pages read here |
| The model you want is a preview model and the workload is business-critical | Do not remove the seam | OpenAI advises against preview models for business-critical workloads unless you can migrate fast |
| You need prosody on the way in but control on the way out | Half-cascade | It buys both, at the cost of operating two models |
Who speech-to-speech is wrong for
Regulated scripts, first and hardest. If a disclosure, a consent line, a price or a medication name must leave the agent exactly as written and be reproducible in an audit, a model that improvises the wording cannot do the job, and no amount of prompting closes that gap because the vendor documentation does not promise it will. The same applies wherever the transcript is the record rather than a convenience: realtime models produce no interim transcripts, so live captioning, transcript-driven routing and real-time supervisor screens all need an STT plugin bolted back on, which quietly reintroduces the component you removed. It is also wrong where tool calling carries the value, and where you need to diff behaviour across a model change. See compliance architecture for AI sales agents in regulated industries for what an audit trail has to contain.
Who the pipeline is wrong for
Genuinely conversational, interruption-heavy, latency-critical work. If callers talk over the agent, change their mind mid-sentence, or are emotional, the pipeline is structurally disadvantaged: it throws away prosody at the speech-to-text step, so the model reasons over flat text and cannot hear that someone is annoyed. It is slower by 100 to 300 ms on LiveKit’s numbers, and that gap is audible at the point where a conversation feels natural. It is also wrong when the team is small: three vendors means three contracts, three rate limits, three status pages and three failure modes, and failover design gets three times harder. A single full-duplex session is genuinely simpler to run, and simplicity is a real engineering value, not a concession. If sub-second response is the product, start at why sub-second voice AI latency matters and work backwards.
The hedge that is actually available, and what it costs
You do not have to pick a winner today. Two documented boundaries let a team change its mind later.
The first is the half-cascade: a realtime model for input understanding paired with a separate text-to-speech for output. You keep prosody on the way in and exact scripts on the way out. The cost is two models to configure instead of one, the full TTS leg back on the bill, and a check that your realtime model supports a text-only response modality, because not all of them do.
The second is delegation. OpenAI documents GPT-Live client delegation as connecting your own agent or workflow using the backend model and provider you choose, with the live model handling only the spoken interaction. That is an abstraction boundary between voice and reasoning drawn by the vendor, and it is why the changelog entry announcing $0.05 per minute also says the backend model is charged separately.
The maintenance cost is the part teams underestimate, and it is not money. It is that a seam you never exercise is not a seam. If the text boundary has never actually carried a different provider, you find out on migration day that the prompt encodes the old model’s quirks, that your evals only ever ran end to end, and that your transcripts are in a shape the replacement does not accept. The fix is cheap: one scheduled job a month that swaps the component behind the seam and replays a fixed scenario set. The same logic decides whether the swap is yours to make at all — private AI deployment for sales agents, where the model runs on customer infrastructure, changes who controls the shutdown date, which is why Zian AI supports it.
How these numbers were counted, and what stayed uncertain
The provider counts are a first-party count of a third-party page: on 14 September 2026 we counted the rows of the plugin tables on the LiveKit Agents STT, LLM, TTS and realtime model overview pages, giving 30, 21, 38 and 9. Nine collapses further on inspection, because three of them are OpenAI surfaces — OpenAI Realtime API, OpenAI GPT-Live and Azure OpenAI Realtime API — alongside Amazon Nova Sonic, Gemini Live API, NVIDIA PersonaPlex, Phonic, SpaceXAI Grok Voice Agent API and Ultravox.
Two things we did not claim. Amazon publishes Nova Sonic token rates on its Bedrock pricing page, but they render client-side from a pricing API, so the AWS lifecycle policy is quoted here instead of a price. And Deepgram annotates its streaming rates as limited-time promotional, so the table carries the regular $0.0077 per minute rather than the $0.0048 showing today.
Frequently asked questions
Is speech to speech always faster than a pipeline?
No, but it starts lower. LiveKit puts a streamed cascaded pipeline at 300 to 600 milliseconds end to end, a speech-to-speech model at 200 to 300 milliseconds, and an unstreamed pipeline at 1,000 to 2,000 milliseconds or worse, which is where most of the gap people report comes from. LiveKit also states a conversation feels natural under one second and broken over two, so a well-streamed pipeline sits inside the natural band. The per-stage breakdown is in the LiveKit guide to sequential pipeline architecture for voice agents: roughly 10 to 50 milliseconds for voice activity detection, about 200 for speech to text, 300 to 800 for the first language model token, and 100 to 200 for the first audio chunk.
Can a speech to speech model read a script word for word?
No. The LiveKit realtime models documentation states that realtime models offer no method to directly generate speech from a text script, that instructed output is not guaranteed to precisely follow a provided script, and that full duplex models cannot speak a script word for word, so scripted speech needs a separate text to speech model. That single fact decides the architecture for regulated scripts, disclosures and quoted prices.
How much notice do I get before a voice model is shut down?
It depends on the lifecycle stage, and the range is wide. OpenAI publishes at least 6 months for generally available models and warns that preview models may be retired with much shorter notice, such as 2 weeks. Google states Gemini preview models will be deprecated with at least 2 weeks notice, and every Gemini Live API and native audio model ID carried the preview suffix as at 14 September 2026. Amazon Bedrock publishes two legacy periods, 6 months and 45 days.
Did OpenAI deprecate its text to speech models as well?
Not as at 14 September 2026. The OpenAI deprecations page lists the legacy audio, realtime and transcription families on 20 July 2026 for shutdown on 20 January 2027, and the transcription models whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe and gpt-4o-transcribe-diarize on 26 August 2026 for shutdown on 26 February 2027. No text to speech model appears in the upcoming or past deprecation lists on that page. We read the full page, not a summary.
What is a half cascade, and when is it the right answer?
A half cascade pairs a realtime model for speech understanding with a separate text to speech model for output: the realtime model returns text and the text to speech model speaks it. LiveKit documents it as the option to choose when you want both realtime speech comprehension and full control over what the agent says. It is right when prosody on the way in and exact words on the way out both matter, and the price is operating two models.
Which architecture should I pick if I am in a regulated industry?
Start from the pipeline and justify any deviation. LiveKit rates the pipeline as the only option with a full text trail at every stage, a realtime model as limited for auditability, and a half cascade as output text only. If your obligation is to reproduce what the agent said, a half cascade is the least you can accept. If it extends to what the caller said, in real time, you need the speech to text stage as well.
Zian AI is in a partnership-application beta. Autonomous agents for outbound appointment setting, support and call closing across phone, SMS, email and WhatsApp, in 30+ languages, with private model deployment available on customer infrastructure. Apply For Partnership