Quick answer
Because a silent leg is not a failed leg. On a half-open WebSocket the socket stays open, every send succeeds and nothing raises, so no exception reaches your retry or your fallback. Two public production timelines: 23.4 seconds with no transcriber event while the caller spoke, and 66 seconds of dead air.
This is a detection problem, not a failover problem. Configuring a second transcriber, voice and model is a different job, already written up in how to build failover into an AI voice agent, and none of that build sequence is repeated here. This page is the case upstream of it: the leg has not errored, closed or timed out. Below is a census of which legs of a LiveKit Agents or pipecat stack can notice a dead socket at all, how long each takes, and the counter that catches what none of them do.
Why is my AI voice agent silent when nothing in the logs is wrong?
A TCP connection can die without either end being told. No FIN, no RST – a NAT entry expires, a load balancer reaps an idle flow, a peer stops reading. To your process the socket is still established, or sitting in CLOSE_WAIT, and from inside an await both look identical to health.
That one fact produces every symptom here. The receive task parks on ws.receive() and never returns. The reconnect loop above it is exception-driven, so it never gets a turn. The audio send keeps succeeding, so your outbound byte counter climbs and your dashboard stays green. The report on the Telnyx transcriber plugin, livekit/agents issue 7358, opened 19 September 2026 and still open, states it in one sentence: _connect_ws opens the socket without heartbeat, so recv_task parks on ws.receive() and the retry in _main_task, which only runs when something raises, never gets a turn.
Your alerting is built on exceptions, and this failure does not produce one.
Three nearby faults are not this one: dead air while the agent waits on a tool call, in why an AI voice agent goes silent on tool calls; a session silently replaced that returns without the conversation, in voice agent context loss on a mid-call reconnect, where the reconnect happens and here it never does; and a call ending at the same duration every time, the opposite signature, in why AI calls always end at the same duration.
Four shapes this takes, ordered by how often they appear in the trackers we read
Method, so you can weigh it: we read 13 issue reports across livekit/agents, livekit/agents-js and pipecat-ai/pipecat filed between 6 August and 19 September 2026, plus the 3 pull requests attached to them, and classified each by mechanism rather than symptom. This is a sample of what we read, not a census of those trackers; the counts are ours, and every state and date was re-checked on 20 September 2026. Our method for reading a tracker, including why a closed issue is not a fixed one, is in auditing a voice platform for leaked keys and PII.
| Shape | In our sample | The check that confirms it |
|---|---|---|
| 1. Inbound went quiet on an open socket. Nothing raises; your logs have a last-transcript timestamp and no line after it. | 5 of 13 | Seconds since the last inbound message on that leg. If it passes your turn budget while audio is still being sent, this is your shape. |
| 2. A local safety timer was disabled by the condition it guards. An open turn or away state pauses the timer that would have caught it. | 4 of 13 | Whether your idle timer is gated on a flag the failure itself sets, and whether it runs at all. In pipecat that is UserIdleController: suppressed while a user turn is in progress, and user_idle_timeout defaults to 0 – off. |
| 3. The outbound send blocked. The peer stopped reading, the kernel send queue filled, the write never returned. | 2 of 13 | ss -tinm on the process: socket in CLOSE_WAIT, bytes stuck in the send queue, retransmissions climbing. |
| 4. The retry budget was spent by ordinary reconnects. Every reconnect succeeded; the counter was only reset by the caller speaking. | 2 of 13 | Count successful reconnects per session, not failures. Both instances were fixed on 16 September 2026. |
Shape 1 is the most common and least visible, which is why it is first. Shape 3 is the rarest and most destructive, because it takes the whole pipeline rather than one leg: in pipecat issue 5796, opened 16 September 2026 the transcriber socket sat in CLOSE_WAIT with 83,514 bytes in the kernel send queue and 11 retransmissions, while the voice socket on the same host and the same library logged a keepalive ping timeout and reconnected 35 seconds into the stall. Same process, same peer, opposite outcome, decided by which direction was blocked.
Which leg of my stack can even notice a dead socket?
We could not find this published for an operator anywhere we looked, so we counted it. The method is stated so you can reproduce the denominator exactly.
Scope. Every module under livekit-plugins/ at commit e5024c76d (main, 19 September 2026) that opens a WebSocket for a speech-to-text or text-to-speech service. grep -rl ws_connect --include=*.py livekit-plugins/ returns 53 files; we excluded 6 that are not a media leg (the OpenAI responses LLM and realtime models, the NVIDIA and Ultravox realtime models, and the Lemonslice and LiveAvatar avatar transports), leaving 47 on aiohttp. We added the 3 modules that use the websockets library – the gnani transcriber and voice plugins and the vakyam voice socket – for a denominator of 50. The speechmatics and palabra plugins are excluded on purpose: they hand the socket to a vendor SDK, so the setting is not in this repository to read.
The client library decides the default. In aiohttp, heartbeat defaults to None and DEFAULT_WS_CLIENT_TIMEOUT carries ws_receive=None, so an aiohttp leg detects nothing unless its author opted in; the websockets library ships keepalive on. All three websockets modules are therefore covered without anyone deciding anything, 23 of the 47 aiohttp modules opted into something, and 24 of the 50 have no in-plugin mechanism at all.
| Mechanism in the plugin | Modules, of 50 | Which ones | Time to detect a socket that goes quiet |
|---|---|---|---|
| Protocol keepalive with a pong deadline | 10 | aiohttp heartbeat=: the deepgram plugin’s stt.py and stt_v2.py (30.0), meta stt.py (30.0), sarvam stt_streaming.py (30.0), sarvam tts.py (20.0), smallestai stt.py (5.0), fishaudio tts.py (30.0). Via the websockets library: gnani stt.py and tts.py (20/20 set explicitly), vakyam _websocket.py (library default). |
1.5 times the heartbeat: 7.5 s at 5.0, 45 s at 30.0. 40 s for the websockets modules. |
| Receive deadline that raises, so the exception-driven reconnect gets its turn | 12 | asyncai, bland, cartesia, respeecher, slng, smallestai, sarvam and the deepgram plugin’s two voice modules pass timeout= to ws.receive() in the streaming loop; both rime adapters do the same; gradium tts.py sets ClientWSTimeout(ws_receive=...) on the connection. |
10.0 s – these pass APIConnectOptions.timeout, whose default is 10.0. The fastest mechanism in the repository apart from one 5.0-second heartbeat. |
| Receive deadline counted and reported | 1 | assemblyai stt.py |
First warning at 15 s, then every 15 s, and it compares the send side to say whether the stall is upstream or downstream. |
| Receive deadline caught and discarded | 4 | baseten stt.py, fireworksai stt.py, gradium stt.py (5 s, then continue); inworld tts.py (60 s, then continue) |
The clock exists and nothing reads it, so: none. |
| No mechanism in the plugin | 24 | includes telnyx on both legs, soniox on both legs, xai on both legs, openai stt.py, elevenlabs on both legs, the cartesia transcriber streams, minimax, murf, neuphonic, resemble |
Unbounded. Nothing in the plugin ends the wait. |
Three things in that table are the finding. First, the spread is six-fold across legs a reader assembles into one call: 7.5 seconds on one transcriber, 45 on another, nothing on 24 modules. Second, a receive deadline beats a heartbeat – the 12 bounded modules detect in 10 seconds, four times faster than the common 30-second heartbeat, which is the reverse of what the tracker discussion assumes, and the two legs of one plugin package are often not alike: the deepgram plugin’s transcriber modules pass a heartbeat while its voice modules bound their receive instead, and telnyx and soniox do neither on either leg.
Third, and worth reading twice: eight of those 24 modules run an application-level keepalive write, and seven prove nothing, because a write into a half-open socket succeeds. The cartesia transcriber streams send a protocol ws.ping() every 30 seconds with no pong deadline anywhere, so the ping leaves and nothing waits for the answer; the elevenlabs, slng and soniox modules send a keep-alive message on a timer. The soniox report, issue 7356, also opened 19 September 2026 and open, spells out why that is worse than nothing: the plugin has a keepalive task, and it catches every exception and returns. The eighth proves the rule – the baseten qwen3_tts.py keepalive waits 3.0 seconds for a reply and drops the socket if none comes, but it guards the idle socket parked between turns, not the one carrying a live turn.
The repository is mid-fix, which decides your version pin. The proposed soniox fix, pull request 7357, adds heartbeat=30.0 and calls it parity with Deepgram and Muse; it was still open on 20 September 2026. The equivalent fix for the deepgram plugin, pull request 7206, merged on 11 September and shipped in livekit-agents 1.8.2 on 15 September 2026. Which plugin you picked decides whether you have this bug, not which framework you picked.
How long a heartbeat actually gives you
aiohttp documents the parameter precisely: a ping every heartbeat seconds, a wait for the pong, and a close if none arrives, with the timer reset on any inbound data. The pong deadline is not in the documentation – it is in aiohttp’s source, where _pong_heartbeat is heartbeat / 2.0. So for a provider that goes quiet just after sending you something:
Detection ceiling = 1.5 times the heartbeat. 45 seconds at 30.0, 7.5 at 5.0. Detection usually lands below that ceiling rather than on it, because the ping timer’s phase relative to the moment of the stall is arbitrary. Then the close raises, the plugin reconnects, and only then can a fallback adapter try a second provider.
Pull request 7357’s own author measured it and reached the same number from the other direction: with heartbeat=30.0 the plugin logs ServerTimeoutError('No PONG received after 15.0 seconds') at 56.7 seconds on a run whose last transcript was at 10.0 seconds, and the author’s summary is that a 30-second heartbeat means roughly 46 seconds to detect, “which is long for a live call”. A 15.0-second pong wait on a 30.0-second heartbeat is heartbeat / 2.0, measured rather than derived. That 46.7-second gap sits just past the 45-second ceiling because aiohttp rounds timeouts of five seconds or more for scheduling, so treat 1.5 times the heartbeat as a planning figure rather than a hard bound.
pipecat uses the websockets library, which defaults to a 20-second ping interval and a 20-second ping timeout: a 40-second ceiling, and the number behind the 35-second detection in issue 5796. One pipecat service turns it off outright – the AWS Transcribe transcriber passes ping_interval=None and ping_timeout=None, so that leg has no protocol liveness check at all. We report the setting, not a judgement. Both ceilings are our arithmetic over published defaults, not a claim either library or any voice vendor makes.
The last-event clock: the test that distinguishes all four shapes
The last-event clock: log two counters per leg – bytes sent since the last inbound event, and seconds since the last inbound event – and alert on the second, never the first.
| Counter | What it measures | What it says during this failure | Use it to |
|---|---|---|---|
| Bytes sent since the last inbound event | That your process is healthy and still streaming audio | Climbing steadily. Everything looks fine. | Separate shape 1 from shape 3. If this stops climbing too, the send side is blocked. |
| Seconds since the last inbound event | That the far end is still answering | Climbing past every threshold you own, unremarked | Fail the leg. This is the only counter that moves when nothing else does. |
Count any inbound message, not just a transcript. That is the lesson of the two retry-budget issues fixed on 16 September 2026, livekit/agents 7208 and livekit/agents-js 2410: the budget was replenished only by a final transcript, making it a counter topped up by the caller speaking rather than by the connection recovering – and the gaps where an agent has no final transcript are the gaps where an idle socket is most likely to be reaped. One reporter’s own log shows three good reconnects and then a fourth idle timeout closing the session after twelve quiet minutes.
Set the clock below the mechanism your plugin already has, or you have built a duplicate: 6 seconds sits inside the 10-second receive deadline, the 40-second websockets ceiling and the 45-second aiohttp one, and would have flagged both timelines at the top of this page. Then set a second, longer threshold on the turn state itself, because an open turn that never closes disables the timers that would otherwise end the call. Do not invent the implementation: the assemblyai transcriber already does exactly this, bounding its receive at 5 seconds, counting consecutive timeouts, warning at 15 seconds and every 15 after, then comparing the time since its last outbound audio frame to say whether the stall is upstream or downstream.
Why a byte counter cannot see this
pipecat’s Deepgram Flux transcriber has a watchdog, and it watches the wrong direction: at commit dbdf21a, _watchdog_task_handler in services/deepgram/flux/stt_base.py fires when we have not sent audio for more than 500 ms, and sends silence so the provider’s turn timers keep running. Nothing watches for the absence of inbound events. The consequence is issue 5735, opened 13 September 2026 and open, whose reporter published a timeline from a production call on pipecat 1.10.0 over a Telnyx WebSocket transport at 8 kHz: a turn opened on background noise at t=13.04 s, then from t=13.04 to 36.4 s there was no further Flux event of any kind although audio kept flowing to the service, the caller spoke clearly at roughly t=28 to 32 s, and hung up at 36.4 s. That is 23.4 seconds spent talking into a transcriber that was not listening, with a watchdog running and every outbound counter healthy. The reporter’s measurements, not ours. When a Flux turn opens and closes in the first place is a separate diagnosis with its own two-field test, in the Deepgram Flux trigger field autopsy.
Why my configured failover fired three turns late, or not at all
pipecat does catch this on the voice leg, and the guard is better than the tracker thread that established it suggests. At the same commit, TTSService counts audio contexts that complete having produced no audio, and on the third in a row pushes a permanent error, which costs the service its usability and lets the switcher fail over. max_consecutive_zero_audio_contexts defaults to 3 and is on by default, and its own documentation names this failure exactly: it catches a provider that “accepts requests and stays silent – an unknown voice ID, say – which no error ever surfaces”. The widely repeated claim that a silent voice provider can never trigger a failover is out of date.
Two things survive it. First, the unit is a turn, not a second. The first two silent contexts push an error the service is expected to carry on from, and the strategy ignores those by design: it returns without switching while the failed service is still usable. So three turns of dead air are the designed cost of the recovery, and the caller hears three attempts at a sentence that never arrives. Run the last-event clock underneath it, not instead of it.
Second, and larger: the machinery is on the voice service and there is no counterpart on the transcriber. A silent transcriber produces no audio context and no completion, because that concept belongs to the voice side; stt_service.py reports an error when transcription raises, and nothing in it counts silence. So the leg behind this page’s headline evidence – a Flux transcriber that kept its socket open and stopped sending events – is not covered by that guard at all. The switcher’s two conditions stand: a non-fatal error frame naming a processor, and a service already marked unusable. A quiet transcriber produces neither.
On the LiveKit side the boundary is narrower than it looks. The fallback adapter’s attempt_timeout, 10.0 seconds by default, goes down as the connection timeout, and in the telnyx and soniox plugins that value wraps the ws_connect handshake only – once the handshake succeeds it is out of the picture for the life of the stream. A provider that fails the handshake or returns an HTTP error is handled well; one that answers and then goes quiet is a different event.
So build the fallback – the configuration is in our failover build guide – but do not let it stand in for detection. The trap when the call finally ends is not that it leaves no code: it is that no ended-reason code distinguishes a silent leg from a caller who simply stopped talking, because a call killed by the silence window reports the configured silence timeout, the same code a genuinely quiet caller earns. The full vocabulary is in what an AI voice agent call-ended reason code means.
What it costs to run this, honestly
The detection work is small: two counters per leg, one shared time source, a threshold. Call it a day. The upkeep is not. You will be reading plugin source at every framework bump, because the census above moved twice in ten days. You need an alert that fires on a metric going flat rather than on a log line appearing – our guide to OpenTelemetry spans for voice agents reached that point before this page did, in the same words: the dangerous failure in telemetry is not an error, it is a panel that is green because the counter never fired. Start there for the wiring, and with AI agent observability for what to put on it. Decide what the agent says when it fails a leg mid-sentence, against the sub-second budget in why voice AI latency matters. And give it an owner: an unowned check rots until it is found by the same silence it was built to catch.
Below a few hundred calls a month on an internal line, log the two counters, set the 6-second clock and stop. Above that, where a silent call is a lost customer, it is a standing commitment rather than a task. That is the honest crossover.
Zian AI is in a partnership-application beta. Our autonomous sales agents run live phone, SMS, email and WhatsApp outreach in 30+ languages, with private model deployment on customer infrastructure for teams that need the media path inside their own boundary. There is no free trial and no self-serve signup. If you are evaluating agentic voice, Apply For Partnership.
Where every figure on this page comes from
| Figure or statement | Who published it | Link (their own page) | Date read |
|---|---|---|---|
Telnyx plugin opens the socket without heartbeat; last transcript at 9.2 s, then nothing for 110 s |
livekit/agents contributor, open issue | issue 7358 | 20 Sep 2026 |
| Soniox keepalive task catches every exception and returns | livekit/agents contributor, open issue | issue 7356 | 20 Sep 2026 |
Proposed soniox fix adds heartbeat=30.0, “parity with Deepgram and Muse”; measured detection at 56.7 s on a run whose last transcript was at 10.0 s; “30s heartbeat means ~46s to detect”. Still open. |
livekit/agents contributor, open pull request | PR 7357 | 20 Sep 2026 |
| The deepgram-plugin fix merged 11 Sep 2026, shipped in 1.8.2 on 15 Sep 2026 | LiveKit | PR 7206, release 1.8.2 | 20 Sep 2026 |
| The census: 50 media modules, 10 / 12 / 1 / 4 / 24, scope and exclusions stated above | Zian AI, count over LiveKit’s source at commit e5024c76d |
livekit-plugins, assemblyai/stt.py | 20 Sep 2026 |
heartbeat semantics and the timer reset; pong deadline heartbeat / 2.0; ws_receive defaults to None and re-raises |
aiohttp | client reference, client_ws.py | 20 Sep 2026 |
ping_interval=20, ping_timeout=20 by default |
websockets | asyncio client reference | 20 Sep 2026 |
APIConnectOptions.timeout defaults to 10.0 |
LiveKit source | types.py | 20 Sep 2026 |
| Ceilings of 7.5 s, 10.0 s, 40 s, 45 s | Zian AI, arithmetic over those defaults; not a vendor claim | Shown in full above | 20 Sep 2026 |
| 23.4 s with no transcriber event; caller spoke at t=28 to 32 s, hung up at 36.4 s | pipecat contributor, open issue, own production call on 1.10.0 | issue 5735 | 20 Sep 2026 |
| 66 s of dead air, 19.5 s to detect | pipecat contributor, open issue, own production call on 1.7.0 | issue 5305 | 20 Sep 2026 |
CLOSE_WAIT, 83,514 bytes queued, 11 retransmissions; voice leg detected at 35 s, transcriber logged nothing |
pipecat contributor, open issue | issue 5796 | 20 Sep 2026 |
max_consecutive_zero_audio_contexts defaults to 3, on by default; third silent context pushes a permanent error; no equivalent counter on the transcriber service |
pipecat source at commit dbdf21a |
tts_service.py, stt_service.py | 20 Sep 2026 |
The two failover-gate conditions; the Flux watchdog watching outbound audio; AWS Transcribe passing ping_interval=None |
pipecat source at commit dbdf21a |
service_switcher.py, flux/stt_base.py, aws/stt.py | 20 Sep 2026 |
UserIdleController is suppressed during a user turn; user_idle_timeout defaults to 0 |
pipecat source at commit dbdf21a |
user_idle_controller.py | 20 Sep 2026 |
| Retry budget reset by a transcript, not a reconnect; three good reconnects then a close at 13:04:48. Both closed 16 Sep 2026. | livekit/agents and agents-js contributors | issue 7208, issue 2410 | 20 Sep 2026 |
| The silence-window ended-reason codes and their per-platform settings | Zian AI, mapped from each vendor’s own documentation on our own page | call-ended reason codes | 20 Sep 2026 |
| Current releases: livekit-agents 1.8.2 (15 Sep), pipecat 1.11.0 (18 Sep 2026) | LiveKit and pipecat release pages | 1.8.2, 1.11.0 | 20 Sep 2026 |
Frequently asked questions
Why did my AI voice agent stop responding mid-call with no error in the logs?
Most often because the WebSocket to your transcriber or voice provider is half-open. With no FIN and no RST it still looks established, every send succeeds, and the receive task waits forever on data that is not coming. Nothing raises, so nothing is logged and no retry fires. Check the time since the last inbound message on that leg.
How long can a dead WebSocket go unnoticed on a voice call?
Anywhere from 7.5 seconds to never, decided by one constructor argument. Where a plugin passes a heartbeat, aiohttp pings every heartbeat seconds and closes if no pong arrives (aiohttp client reference, read 20 September 2026), and the pong deadline in its source is half the heartbeat, so the ceiling is 1.5 times the setting. A receive deadline is faster, typically 10 seconds. Where neither is set, nothing ends the wait.
Does a keepalive message prove my transcriber connection is alive?
No. A keepalive message is a send, and a send into a half-open socket succeeds. A public report on the Soniox plugin in livekit/agents, opened 19 September 2026, records both halves: the plugin has a keepalive task, and it catches every exception and returns. Only a mechanism that waits for a response tells you anything.
Why did my fallback transcriber not take over?
Because the switch is gated on an error and a silent leg does not raise one. pipecat now catches the equivalent failure on the voice side, counting audio contexts that complete with no audio and writing the service off on the third, but there is no counterpart on the transcriber service.
Is a call that goes silent the same problem as a call that drops?
They are diagnosed from opposite ends. A dropped call leaves a terminal code you read afterwards; a silent leg leaves either no code, because the caller gave up, or the ordinary silence-timeout code, which is what a genuinely quiet caller earns too. No published ended-reason code separates the two, so the reliable signal is a counter logged during the call.
Will upgrading my framework fix this?
Partly, and only for the plugin you use. The deepgram-plugin fix for this mechanism merged on 11 September 2026 and shipped in livekit-agents 1.8.2 on 15 September, while the telnyx and soniox reports opened on 19 September and the proposed soniox fix was still open on 20 September. Upgrading moves the census; it does not complete it.
Running agents on a line where a silent call is a lost customer? Zian AI runs autonomous phone, SMS, email and WhatsApp agents with SmartReach AI™ and PrecisionPitch AI™, CRM write-back through HubSpot, Salesforce, HighLevel and Zapier, and private model deployment on customer infrastructure. We are in a partnership-application beta, with no free trial and no self-serve signup. Start with the Zian AI FAQ on autonomous AI sales agents or read how we deliver on the Zian AI solutions page, then Apply For Partnership.