Quick answer
An agent that re-greets a caller mid-call has lost its in-call context. Two causes: the realtime session was replaced and handed back as text only, or a summariser compacted the instructions away. Detect the first from one number. In livekit/agents-js issue 2346, per-turn audio input tokens fell 2,758 to 351 while text held at 17,210.
Why does my AI voice agent greet the caller again in the middle of a call?
Because something between your application and the model replaced the conversation, and the transport did not tell you. The reported case, on @livekit/agents 1.6.4 with gemini-3.1-flash-live-preview, a Dutch (nl-BE) voice agent over SIP: “During a normal conversation the agent occasionally restarts as if the call had just begun: it greets the user again, mid-sentence, and invents details it cannot know. The conversation history is gone from the model’s point of view, while the WebSocket itself never reported an error to the application.” The agent also read the clock wrong, 18:40 against a prompt that supplied 21:04. Frequency as reported: twice across 68 turns in three days of production traffic.
That is why it survives testing. A dropped call is loud; this is silent. The socket is healthy, the audio flows, the caller keeps talking, and only the model has been reset. The two causes also sit in different places. The first starts on the provider side, where a connection ends and takes the session with it (Google Gemini Live API, OpenAI Realtime API) — but the lost context is usually your own replay rather than theirs: issue 2346’s reporter traced it to the framework’s provider-format conversion, which builds parts from string content only, so the audio turns are dropped on the way back and a native-audio model is handed a text-only transcript. The second is an over-eager context summariser on the framework side (LiveKit Agents, Pipecat).
Two boundaries, so you do not read the wrong page. If the call ends at a repeating number of seconds, that is a timer and the constant names the layer: see our duration lookup table for calls that always end at the same duration. If you want recall across calls, our guide to voice agent memory between calls owns that: in its four-memory model it is layers 3 and 4, and this page is layer 1, the turn context inside one live call. Nor is this failover configuration — nothing appeared to fail.
The Audio-Token Collapse Test: the signal that a session was replaced
The rule: within one realtime session, and with no server-side context compression running, cumulative audio input tokens only ever rise. When that count falls below a quarter of its trailing three-turn median while text input tokens stay within 10% of theirs, the session you are talking to is not the session you started.
The raw material is the reporter’s own per-turn table in issue 2346, reproduced as published. It is their production data from a Gemini Live session on 25 August 2026, not ours.
| Turn time (UTC) | text_in |
audio_in |
Trailing audio median | Test result |
|---|---|---|---|---|
| 19:04:01 | 17,707 | 2,016 | no history yet | no verdict |
| 19:04:17 | 17,754 | 2,367 | no history yet | no verdict |
| 19:04:36 | 17,816 | 2,758 | no history yet | no verdict |
| 19:04:52 | 17,210 | 351 | 2,367 | fires: audio at 14.8% of median, text at 96.9% |
Audio fell 87.3% in one turn; text fell 3.4%. A quiet caller cannot produce that: saying less makes a cumulative count grow more slowly, and this count fell below its own first-turn value. Our own worked model of realtime voice cost per minute shows the same accumulating shape from the billing side, because the whole conversation is re-billed on every turn. The asymmetry between the two modalities is the whole test, and it needs no new telemetry vendor.
The one exclusion. A cumulative audio count can legitimately fall when server-side compression fires, so the test is valid only when you know whether compression is on. In the reported case it had not fired on published defaults: Google’s Live API reference says triggerTokens, “If not set, the default is 80% of the model’s context window limit”, and the Live API capabilities guide puts the window at 128k tokens for native audio output models and 32k for others. The turn before the collapse carried 17,816 + 2,758 = 20,574 tokens; reading 32k as 32,768, 80% of the smaller window is 26,214, so the trigger was not reached on either.
If you already run traces you do not need a second pipeline. The convention attribute is gen_ai.usage.audio.input_tokens, and our guide to OpenTelemetry spans for voice agents records that from LiveKit Agents 1.8.0 realtime inference moved onto a realtime_inference child span, so a query still reading realtime token usage from agent_turn returns nothing.
Cause two: a summariser that deletes my instructions on message 21
The second cause needs no network event at all. A framework-side compactor summarises the history and the instruction block goes with it.
Read from Pipecat’s released source at tag v1.11.0 on 20 September 2026, LLMAutoContextSummarizationConfig ships max_context_tokens = 8000 and max_unsummarized_messages = 20, and get_messages_to_summarize preserves the first message only when its role is exactly system, setting summary_start = 1 if first_is_system else 0. Send your instructions with the developer role instead of system, and summary_start is 0, so the instructions sit inside the summarised range. Pipecat issue 5595, raised 2 September 2026 against 1.8.1 and still open on 20 September 2026, reports the production consequence: the assistant continued after the 21st message with no instructions at all, replying with a bare greeting. The same issue notes that a prompt built from input_text parts counts as roughly zero tokens, so the token trigger never fires and only the message counter does. Our guide to caller memory between calls already cites this issue from the other side, as a caveat about injected caller memory being summarised away; that reading is ceded to it, and what this page adds is the check on the current release — the preservation test at v1.11.0 is still the one the issue describes at 1.8.1.
Telling the two causes apart takes one pass through the logs.
| What you observe | Session was replaced | Summariser compacted |
|---|---|---|
| Audio input tokens | Collapse in one turn, text steady | Both modalities fall together |
| When it happens | Clock time: near a connection lifetime | Message count: at the configured threshold |
| Reproducible on demand | Only by killing the socket | Yes: hold a call past the threshold |
| Transport reported an error | No | No |
| First thing to change | Restore history explicitly on the new connection | Move instructions out of the compacted window |
Pipecat issue 5465, open since 27 August 2026, is the same family from the other direction: text, audio and tool results dropped while a Gemini Live service reconnects, and, in the reporter’s words, “The caller has no idea it didn’t go out.”
What each provider tells me before it takes the session away
Every figure below is from that provider’s own documentation, read on 20 September 2026. The wider catalogue is not ours and we are not restating it: Zylos published a survey of session resumption and continuity in July 2026 covering caps and keepalive cadences across OpenAI, Gemini, Nova Sonic, Deepgram, Vapi and Retell. What it does not carry, and we string-searched a saved copy before saying so, is a detection signal for a socket that is healthy while the context is gone.
| Provider | Published session ceiling | Connection lifetime | Warning before the cut | Handle to resume |
|---|---|---|---|---|
| Gemini Live API (Google) | 15 minutes audio-only, 2 minutes audio-video, without compression | “around 10 minutes”, and when the connection terminates the session terminates as well | GoAway carrying timeLeft. No fixed value published: it “will never be less than a model-specific minimum, which will be specified together with the rate limits for the model” |
Yes. Resumption tokens “are valid for 2 hr after the last sessions termination” |
| OpenAI Realtime API | “The maximum duration of a Realtime session is 60 minutes” | Not stated on the realtime conversations guide | Not stated on that guide | No resumption mechanism described on that guide |
The connection is shorter than the session. Google documents a 15-minute audio session ceiling and a connection lifetime of around 10 minutes, so a 20-minute sales call crosses at least one connection boundary before it reaches any session cap. That is the window in which issue 2346 fired.
Nothing is on until you turn it on. Google’s Live API reference lists sessionResumption and contextWindowCompression as Optional setup fields and makes each behaviour conditional: “If included, the server will send SessionResumptionUpdate messages”, and “If included, the server will automatically reduce the size of the context when it exceeds the configured length”. The default is that neither mechanism is doing anything for you.
How do I set this up so a 20-minute call survives a reconnect?
Day one, and it costs nothing. Log per-turn input tokens split by modality, with the connection identifier beside each turn. Day one is done when you can print a table like the one above for any call you took today. Whether you can is itself the finding of the first afternoon.
- Split the token log by modality. Audio and text input counts on separate fields, per turn, per connection. A single total hides the signal.
- Add the Audio-Token Collapse Test as an assertion, not a dashboard. Three turns of history, then alarm on audio below 25% of the trailing median with text within 10%. Confirm the cumulative shape on your own stack first: field names differ by framework, and a genuinely per-utterance counter needs a different rule.
- Decide in writing whether you run with a resumption handle. If you do, store the newest handle each time the server sends one and pass it on the next connection. If you do not, you have chosen to rebuild context yourself, which is legitimate but is not a default.
- Restore history explicitly rather than trusting a replay. Google documents
historyConfig.initialHistoryInClientContent: set it true and “after sendingsetupComplete, the server will wait and at first processclientContentmessages untilturnCompleteis true. This initial history will not trigger a model call and may end with role MODEL.” That is the supported seam for putting the conversation back before the caller speaks again. - Move the instructions out of the compacted window. Either put them at
messages[0]with rolesystem, or use the provider instruction field. Pipecat’s own configuration docstring says why the second is safer:system_instruction“is sent on every inference but is not part of the context, and summarization cannot compress it.” - Assert the instruction is present every turn. Cheapest version: a fixed token near the start of the rendered request, checked before the call to the model. Missing means you have found your cause before a caller does.
- Force the failure in staging. Kill the WebSocket mid-call while a caller is speaking, and separately hold a test call past your message threshold. Both are reproducible on demand; neither appears in a happy-path test.
The finish state. A 20-minute call crosses a forced socket reconnect and passes 25 turns without the agent re-greeting, without inventing a fact it was never told, and with the collapse alarm firing within one turn when you deliberately kill the socket. That last clause is the part people skip, and a detector that never fires in staging is not a detector. Budget the restore as latency: putting history back sits on the critical path before the agent can speak, on the same budget as any mid-call knowledge lookup.
Keep this in-house, or hand the realtime layer over
The method above is complete and runnable, and on a single stack it is roughly a day. The build is not what costs you; the tracking is. So the table states the crossover conditions and leaves the conclusion where it belongs.
| Condition | Do it yourself | Hand the stack over |
|---|---|---|
| Typical call length | Under 5 minutes, inside every published ceiling | Over 10 minutes, so every call crosses a connection boundary |
| Realtime providers in production | One | Two or more, each with its own resumption semantics |
| Access to per-turn token counts | Your framework emits them and you can log them | Your platform reports call totals only |
| Who can change the compaction trigger | It is a value in your own configuration | It is a vendor default you cannot read |
| Framework churn you can absorb | You track releases: @livekit/agents reached 1.9.0 on 15 September 2026, three weeks after issue 2346 was raised against 1.6.4 |
Nobody owns upgrade regressions |
The fourth row decides it more often than the rest. A compaction trigger you can read is a five-minute fix; one you cannot read is a support ticket and a fortnight of guessing. That is why the deployment boundary matters more here than any single feature: Zian AI supports private model deployment on customer infrastructure, which puts the summariser threshold and the resumption configuration in files you can open. If you are still choosing the layer you build on, our comparison of LiveKit Agents and Pipecat covers where each keeps the context object.
Where every figure on this page comes from
| Figure or quotation | Who published it | Link (their own page) | Date read |
|---|---|---|---|
Per-turn token table, the re-greeting quotation, the 18:40 against 21:04 clock error, twice across 68 turns, on 1.6.4 and gemini-3.1-flash-live-preview |
Reporter of livekit/agents-js issue 2346 (open, raised 25 August 2026) | github.com/livekit/agents-js/issues/2346 | 20 Sep 2026 |
Instructions summarised away on the message trigger; input_text parts counted as roughly zero tokens |
Reporter of Pipecat issue 5595 (open, raised 2 September 2026) | github.com/pipecat-ai/pipecat/issues/5595 | 20 Sep 2026 |
| Text, audio and tool results dropped during a reconnect, and the caller not knowing | Reporter of Pipecat issue 5465 (open, raised 27 August 2026) | github.com/pipecat-ai/pipecat/issues/5465 | 20 Sep 2026 |
max_context_tokens = 8000, max_unsummarized_messages = 20, the system preservation test, and the system_instruction docstring |
Pipecat, released source at tag v1.11.0 | llm_context_summarization.py at v1.11.0 | 20 Sep 2026 |
15 minutes audio-only, 2 minutes audio-video, connection lifetime around 10 minutes, GoAway, tokens valid 2 hr |
Session management with Live API | 20 Sep 2026 | |
Optional and conditional setup fields, triggerTokens default of 80% of the window, no fixed timeLeft value, historyConfig.initialHistoryInClientContent |
Live API reference | 20 Sep 2026 | |
| Context window of 128k tokens for native audio output models, 32k for other Live API models | Live API capabilities guide | 20 Sep 2026 | |
| “The maximum duration of a Realtime session is 60 minutes” | OpenAI | Realtime conversations guide | 20 Sep 2026 |
| Per-provider session caps and keepalive cadences, ceded to its publisher rather than restated | Zylos | Session resumption and conversation continuity in realtime voice AI agents | 20 Sep 2026 |
@livekit/agents 1.9.0 released 15 September 2026 (the 1.6.4 in issue 2346 is the reporter’s own stated version) |
LiveKit | github.com/livekit/agents-js/releases | 20 Sep 2026 |
Frequently asked questions
Why does my AI voice agent greet the caller again in the middle of a call?
Two causes produce that one symptom. Either the realtime session was replaced and the model was handed the conversation back as text only, so it does not recognise an ongoing conversation, or a summariser compacted the history and removed the instructions. The first shows as a fall in per turn audio input tokens while text input tokens hold steady; the second shows on a message count.
How do I detect that a voice agent session was silently replaced mid call?
Log input tokens per turn, split by modality, and watch for a fall. In the LiveKit JavaScript agents repository, issue 2346, the audio input count rose across three turns (2,016, then 2,367, then 2,758) and then fell to 351 while text input stayed above 17,200, and the WebSocket reported no error. With no server side compression running, a cumulative audio count cannot shrink while one session continues, so a fall below a quarter of its trailing median with text steady is the signal.
Does Gemini Live restore the conversation when I reconnect with a resumption handle?
Only if you configured it. Google documents the handle and says resumption tokens are valid for 2 hr after the last session termination, but the Live API reference lists both sessionResumption and contextWindowCompression as Optional setup fields whose behaviour applies only if included, so neither is active until you set it. Google sets this out in Session management with Live API, read on 20 September 2026.
What is the default that makes a summariser delete my system prompt?
In Pipecat, automatic context summarisation triggers at 8,000 context tokens or 20 unsummarised messages, both read from the released source of version 1.11.0 on 20 September 2026. The utility preserves the first message only when its role is exactly system, so instructions sent with the developer role fall inside the summarised range and disappear.
Will a 15 minute session cap show up as a call that ends or as an agent that forgets?
As a call that ends. Without compression, Google limits audio only sessions to 15 minutes and audio plus video sessions to 2 minutes, and exceeding those limits terminates the session and therefore the connection. An agent that forgets while the call keeps running is the other failure, and it arrives sooner because Google puts the connection lifetime at around 10 minutes, shorter than the session ceiling.
Can I use my existing OpenTelemetry traces for this instead of new logging?
Yes, if your framework emits token usage per turn. The convention attribute is gen_ai.usage.audio.input_tokens, and on LiveKit Agents 1.8.0 and later realtime inference moved onto a realtime_inference child span, so a query that still reads realtime token usage from agent_turn returns nothing.
Build agents that keep the thread on a twenty minute call
Zian AI builds autonomous AI sales agents with live phone, SMS, email and WhatsApp outreach across 30+ languages, with SmartReach AI™ orchestrating message, channel and timing and PrecisionPitch AI™ split-testing the approach. Private model deployment on customer infrastructure puts the summariser threshold and the resumption configuration in your own files. Zian AI is currently in partnership-application beta.