Voice Agent Slow After Transfer: p95 Looks Fine - Zian AI

Voice Agent Slow After Transfer: p95 Looks Fine

Most likely your dashboard is not measuring the turn your callers are complaining about. Until livekit-agents 1.8.1, released 10 September 2026, e2e_latency was never set on the first agent turn after a handoff — reliably the slowest turn in the call. Check your SDK version before you tune anything.

Why my p95 looks fine but callers say the agent is slow after a transfer

Two different things get called a transfer, and only one of them is a measurement problem.

The first is a telephony transfer: the agent places a SIP REFER or dials a human and bridges the legs. When that goes wrong the symptom is usually silence, hold music with nobody on it, or a call that never connects — covered in why an AI voice agent transfer to a human does not connect, and the design question of what the human must receive is the handoff packet.

The second is an in-session agent handoff: a function tool returns a second agent and the session swaps which agent is answering. The call never leaves the platform. The caller hears one long gap and then a different personality. This page is about that second case, and specifically about why the gap the caller hears does not appear in your numbers.

In LiveKit Agents, e2e_latency is computed from the preceding user turn metrics, which were held by the AgentActivity that handled that user turn. A handoff creates a new activity, so the incoming agent first generation started with no user metrics at all and the branch that sets the field never ran. That is the report in livekit/agents issue 7157, filed 7 September 2026 against livekit-agents 1.7.1 and closed as completed on 9 September 2026 (status re-checked 18 September 2026).

The issue states the consequence in one line: a post-handoff turn also pays for the tool call and the new agent context build, so it is reliably the slowest turn in a call, and dropping e2e_latency there biases every latency aggregate optimistically. It was not only the field on the message. The OpenTelemetry histogram lk.agents.turn.e2e_latency and the lk.e2e_latency span attribute are set from the same report, so a dashboard built on either omitted post-handoff turns entirely rather than showing them as slow.

The two-clocks check: is the transfer slow, or is it invisible?

This is the test that separates the two causes, and it needs no code change and no deploy. Take one recorded call that a caller complained about. From the audio alone, mark the timestamp where the caller stops speaking and the timestamp where the first agent word plays. That interval is your audio clock. Then pull what your dashboard reports for that same turn. Compare the two.

What the dashboard reports for the post-handoff turn What it means What to do next
No latency value at all for that turn, while earlier turns in the same call have one Measurement gap. Your percentile is computed over a subset that excludes the turn being complained about Check the SDK version against the table below, then re-baseline
A value that matches the audio clock within roughly 200 ms The transfer really is slow and you are seeing it honestly Work the pipeline: endpointing, time to first token, time to first byte
A value well below the audio gap, or two short turns where the caller heard one The turn is split or the wait is unattributed. Nothing is lying, the span shape is wrong Upgrade to 1.8.2 and read the spans, not the summary
A value well above the audio gap An inline task reported a whole sub-conversation as one latency — the second defect PR 7167 fixed in 1.8.1 Check whether a tool ran an inline task inside that turn, then check the version

Name it and it survives the handover to whoever is on call: the two-clocks check — one call, one turn, the audio clock against the dashboard clock. A dashboard that cannot be reconciled with a recording on a single turn cannot be trusted on a thousand.

Which livekit-agents version measures what

The fix landed between two releases that most teams upgraded straight past. Pull request 7167, which closes issue 7157 and supersedes an earlier community attempt in 7161, was merged on 9 September 2026 and appears in the 1.8.1 release notes. The separate telemetry work shipped in 1.8.2 on 15 September 2026. Release dates below are the GitHub release publication dates for the livekit-agents tags, read 18 September 2026.

Version Released The agent turn that follows a handoff Turn shape in traces
1.7.1 and earlier 27 August 2026 No e2e_latency on the message, no lk.e2e_latency span attribute, no histogram sample A reply that calls a tool renders as two agent_turn spans; the endpointing wait is an empty gap
1.8.0 5 September 2026 Same gap. Also the release that renamed GenAI content events, so older panels can be empty for a second reason Unchanged
1.8.1 10 September 2026 Reported. The session keeps the latest user turn no speech has answered, so the first reply after a handoff claims it (PR 7167) Unchanged
1.8.2 15 September 2026 As 1.8.1 One agent_turn span per speech handle (PR 7143); an eou_wait span back-dated to the moment the caller stopped speaking, so the endpointing wait is no longer an empty gap (PR 7130); an update_agent span around the agent handoff itself (PR 7137)

The decision rule that falls out of this table: if you are below 1.8.1, every post-handoff turn in your history is unmeasured, and no amount of re-slicing the existing data will recover it. You do not have a slow-turn problem in your dashboard, you have an absent-row problem, and the two are fixed differently. What each span is for, and which backend can read it, belongs to the OpenTelemetry span map for voice agents; this page stops at whether the row exists.

Four more ways a green dashboard hides a slow turn

The handoff gap is the one with an issue number, but it is not the only documented absence. The condition in each row below is LiveKit stated behaviour, quoted from its own data-collection reference as read on 18 September 2026; the third column is our reading of what that does to a dashboard, not LiveKit wording. These are design boundaries, not bugs, and each empties a panel rather than erroring.

What goes missing The condition Why it flatters your p95
llm_node_ttft and tts_node_ttfb Only populated by the STT-LLM-TTS pipeline; empty when using a realtime model Your stage breakdown is blank on exactly the architecture people adopt for speed
EOUMetrics entirely Not emitted when using server-side turn detection The endpointing wait is the largest single component of a slow turn and it is not in the data
speech_id on LLMMetrics Not present for proactive agent responses, tool-call follow-ups, or other completions not tied to a user speech turn A dashboard that joins per-stage metrics on speech_id drops every completion on that list; LiveKit does not say whether a greeting spoken from on_enter after a handoff is one, so test your own build
playback_latency Near zero unless a remote avatar worker is in the chain and reports playback via an RPC Playout time looks free

That third row is the durable one. Upgrading to 1.8.1 restores the field on the message, but if your own aggregation joins the stage metrics by speech_id, a proactive reply still has nothing to join on. The version fixes the SDK. It does not fix your query.

Your p95 is a percentile of turns; your caller heard one call

Two pieces of arithmetic, both of which you can run on your own numbers today. They are arithmetic on stated assumptions, not a measurement of anything.

One: what the missing rows do to the number. Take a sample of 100 agent turns in which 12 followed a handoff and were therefore dropped. Your dashboard computes p95 over the 88 turns it can see, so roughly 4.4 turns sit above the reported value. If every dropped turn is slower than that value — the worst case, and the direction the issue says the bias runs — the true count above it is 4.4 plus 12, or 16.4 of 100. The number on the wall labelled p95 is then really the 84th percentile, and if only some of the dropped turns exceed it the truth sits between the 84th and the 95th. Substitute your own handoff rate: the reported percentile is approximately 100 minus (5 percent of the measured turns plus every dropped turn that exceeds it), as a share of all turns.

Two: what a percentile of turns means to one caller. If 5 percent of turns exceed your p95 and a call contains 12 agent turns, the chance that call contains at least one turn above it is 1 minus 0.95 to the power of 12, which is 46 percent. At 20 turns it is 64 percent, and at 40 turns 87 percent. Turns within a call are correlated rather than independent, so treat these as an illustration of the shape and not a prediction. The shape is the point: a per-turn percentile that looks excellent is fully compatible with most callers hitting at least one bad gap.

The band that gap has to fit inside, and why the average is the wrong statistic for any of this, is set out in the sub-second voice AI latency budget. If the slow turn turns out to be a lookup rather than the handoff itself, mid-call retrieval latency is the page for that.

What to measure instead, and what it costs to run

The complete method, in the order it is worth doing:

  1. Record the denominator. Every day, count assistant turns that answered a caller and count how many carry a latency value. Alert on the ratio, not on the latency. This catches the next absent metric as well as this one.
  2. Tag every turn with the agent that spoke it and with whether a handoff occurred in that turn. Without that tag you cannot ask the only question that matters here, which is whether post-handoff turns differ from the rest.
  3. Alert on the worst turn per call, not the percentile across turns. One number per call, the maximum gap, then a percentile over calls. That is the statistic your caller actually experienced.
  4. Run one synthetic call a day that forces a handoff and assert the metric exists. A panel that has never gone red is not evidence that nothing is wrong.
  5. Reconcile against audio monthly with the two-clocks check on a random call.

Then the honest part: none of that is free. It is a day or two of engineering to instrument, a recurring trace-storage bill, and a standing job for whoever owns the dashboard, because the framework will not do the reacting for you. Issue 6982, which proposes configurable latency budget policies so an application can change behaviour when a conversational target is exceeded, was raised on 25 August 2026 and is still open as at 18 September 2026: threshold evaluation, correlation and callbacks remain something you build. That is the arithmetic a team should do before deciding whether voice observability is a thing they want to own.

Situation Do it yourself Hand it over
Under roughly 500 calls a month, one agent, no handoffs Yes. The per-turn fields plus a spreadsheet are enough No
Multi-agent workflows with handoffs, or a regulated transcript Only with a named owner for the dashboard Worth pricing
You cannot answer what the SDK version is in production Fix that first; it is a five-minute answer Not yet

Zian AI builds autonomous sales agents for live phone, SMS, email and WhatsApp outreach in 30 or more languages, and supports private model deployment on customer infrastructure, which is the arrangement teams ask for when call data is not allowed to leave their own environment. Zian has been running outbound acquisition since 2017. If you want the measurement question answered as part of the platform rather than as a project of your own, Apply For Partnership.

Frequently asked questions

Does upgrading to livekit-agents 1.8.1 make the transfer faster?

No. Version 1.8.1, released on 10 September 2026, restores the e2e_latency field on the first agent reply after a handoff, which was absent in every earlier release. The turn takes exactly as long as it did before. What changes is that the number enters your percentile instead of being dropped from it.

How do I check whether the turn after a handoff is measured at all?

Count the denominator. For one call, count the assistant turns that answered a caller, then count how many of those carry a latency value. If the two numbers differ, your percentile is computed over a subset, and the missing turns are usually the slow ones.

What does e2e latency actually measure in LiveKit Agents?

LiveKit documents e2e_latency as the time in seconds from when the user stopped speaking to when the agent began responding, reported per turn on the assistant message, alongside llm_node_ttft, tts_node_ttfb and playback_latency. The field list is in the LiveKit data collection reference.

Is a slow turn after a transfer always the model?

No. The endpointing wait, the tool call that triggered the handoff, the larger set of instructions the incoming agent carries and any lookup it runs all sit inside the same gap the caller hears. Measure the parts before you change the model.

Can I measure this without changing my agent code?

Yes. Take one call recording, mark the moment the caller stops speaking and the moment the first agent word plays, and compare that interval with what the dashboard reports for the same turn. Two clocks, one turn, no deploy.

Related Blogs

Related from Zian AI