“My AI voice agent keeps interrupting itself” - Zian AI

“My AI voice agent keeps interrupting itself”

Your agent is almost certainly not interrupting itself: something in the inbound audio is crossing a speech threshold while it talks, and one test tells you which. On LiveKit Agents 1.8.1 with Silero VAD the stock barge-in gate is min_duration 0.5 s and min_words 0, the false-interruption timer 2.0 s, and AEC warmup 3.0 s — None on outbound SIP.

“My agent cuts itself off mid-sentence and thinks the caller interrupted when nobody spoke”

That is how people write it, in one long sentence: my agent cuts itself off mid-sentence and thinks the caller interrupted when nobody spoke. LiveKit Agents and Pipecat both call it a false interruption. While the agent speaks, a voice-activity model — usually a Silero ONNX checkpoint — watches the inbound audio for the caller taking the floor; something scores as speech, the barge-in gate opens, the sentence stops, the turn is marked interrupted. The prompt did not do this, and it happens on a WebRTC session and a SIP phone leg alike.

It is the mirror image of the agent talking over a caller: if the agent cuts the caller off mid-number, that is end-of-turn detection, covered in why an AI voice agent interrupts callers. If it goes quiet and never returns, that is a third fault again — silence during tool calls.

Is this as urgent as it feels, and what to do in the next hour

Mostly, no. A false interruption is recoverable by design: on LiveKit Agents resume_false_interruption defaults to True and the agent picks the sentence back up after false_interruption_timeout, 2.0 s of silence by default. The caller hears a stumble, not a dropped call. If your own logs put it at two or three per hundred calls, that is a quality problem you can schedule rather than an incident; we have no rate of our own to offer you.

It is urgent in two shapes: when it fires inside the opening greeting, and when the recovery eats the caller reply instead of resuming cleanly, so what the caller said is nowhere in the transcript. That second one reads as being ignored, and has open issue numbers against it in two frameworks.

In the next hour, and none of this needs a vendor: pull the structured turn log for five complaint calls, not the audio, and read one field — whether a caller transcript is attached to the interrupted agent turn, and what it says. A real transcript, a one-word transcript, a phantom transcript such as a bare thank-you, and none at all are four different bugs. Then run the ladder, and touch no threshold before you have.

The silent-line ladder: four tests, and the first one that fires is your cause

The silent-line ladder gets you from “it is interrupting itself” to one cause in four steps, stopping at the first that fires. It needs a test number and twenty minutes. Run it on the same route and codec as the failing calls: the answer differs between a WebRTC session and a phone leg.

  1. Silent line, agent speaking. Put the agent on a call with nobody on the far end — a handset you answer and mute will do — and let it run a sixty-second monologue. Any speech event or committed transcript came from the line or from the agent, which eliminates the caller. Go to step 2. Nothing at all: skip to step 3.
  2. Silent line, agent muted. Same call, agent silent for sixty seconds. Events still firing means the line: comfort noise, background conversation, a noisy trunk. Events only while the agent speaks means the echo path, your own audio arriving back on the inbound leg. Identical on a recording, opposite fixes.
  3. Real calls, read the attached transcript. Real caller audio is opening the gate. If the interrupted turns carry a one-word or two-word transcript — yeah, okay, mm-hmm — the gate is too aggressive and min_words is your dial, not the VAD.
  4. Measure the stop-to-resume gap. Agent stops, waits about two seconds, resumes, and the caller words in that window are missing from the committed transcript: not a threshold, but turn-state cleanup, which is open framework work.
Cause The one observation that confirms it What eliminates it
Echo of the audio the agent is playing Speech events fire on a silent line only while the agent speaks The same events fire with the agent muted
Line or background noise Speech events fire on a silent line with the agent muted Sixty silent seconds produce nothing
Barge-in gate too aggressive Interrupted turns carry a real one-word or two-word caller transcript No transcript, or a transcript nobody said
VAD model or checkpoint sensitivity Phantom transcripts on audio with no speech in it, reproducible in an A/B/A on the checkpoint alone Swapping the checkpoint back changes nothing
Turn-state and stream fragmentation Agent stops, resumes after about 2 s, and the caller words are absent from the transcript The transcript is complete and correctly timestamped

That order is the ladder, not a measured incidence rate; we have no frequency data of our own and will not invent one. Reorder it once you have counts from your own calls, and fold it into the pre-launch pass in how to test an AI voice agent before go-live — a silent-line call is the one test an unscripted human tester can never perform.

Why echo is the cause almost nobody checks

Step 1 exists because of a blind spot: everyone tunes thresholds, almost nobody proves what the microphone is receiving. If your agent output returns on the inbound leg at any usable volume it is a full-fidelity copy of a human voice, and no activation threshold will classify it as silence. Turning the VAD down just makes the agent deaf to real callers too.

The frameworks treat this as a live risk. LiveKit Agents carries aec_warmup_duration on AgentSession, whose documented purpose in the agent session source, read 14 September 2026, is to stop the agent being interrupted by echo before acoustic echo cancellation is ready. It defaults to 3.0 s, and inbound audio is withheld from speech-to-text for that window. The detail worth knowing: the same file sets it to None for outbound SIP calls unless you pass a value explicitly, so on an outbound phone call that protection is off by default.

livekit/agents issue 6030, opened 9 June 2026 and still open as at 14 September 2026, reports background noise and conversation on a SIP leg read as the user speaking, on a session already running a telephony noise-cancellation model. If you are still deciding how audio reaches the agent, whether you need a SIP trunk for an AI voice agent is the upstream question.

The dials, their stock values, and which way to move them

Every value below was read from the owning project source on 14 September 2026: LiveKit turn.py, the Silero plugin vad.py and Pipecat vad_analyzer.py, at livekit-agents 1.8.1, livekit-plugins-silero 1.8.1 and pipecat-ai 1.10.0. Change one dial at a time and re-run the ladder.

Dial Stock value Move it when What it costs you
interruption.min_words (LiveKit) 0 Interrupted turns carry one-word transcripts. Raise to 2 or 3. Nothing; it applies only while the agent speaks.
interruption.min_duration (LiveKit) 0.5 s Short bursts stop the agent. Raise to 0.8–1.0 s. A genuine barge-in takes that much longer to register.
interruption.backchannel_boundary (LiveKit) (1.0, 1.0) s Backchannels stop the agent in the first or last second of a turn, and the adaptive detector is in use. Widen it. Only speech the adaptive detector classes as a backchannel is suppressed; events flagged as interruptions still pass through.
interruption.false_interruption_timeout (LiveKit) 2.0 s Recovery feels slow. Lower toward 1.0 s. None disables resume. Resumes over a caller who was only pausing.
aec_warmup_duration (LiveKit) 3.0 s, None on outbound SIP The clip lands in the opening seconds of an outbound call. The caller cannot interrupt during the warmup.
Silero activation_threshold 0.5 Noisy trunk, after echo is ruled out. Raise to 0.6–0.7. Soft speech missed; deactivation tracks at max(threshold − 0.15, 0.01).
Silero min_speech_duration 0.05 s Sub-100 ms bursts open turns. Raise to 0.1–0.2 s. Single-syllable answers can be dropped.
Silero min_silence_duration 0.55 s Leave it. Sets the endpointing floor, not the barge-in gate. Slows every turn without touching self-interruption.
Pipecat VADParams.confidence 0.7 Echo-prone leg, once the ladder points at the line. Raise toward 0.8. Quiet callers stop registering as speech.
Pipecat VADParams.min_volume 0.6 The intruding audio is quieter than a real caller. Callers on a poor connection fall below the gate.
Pipecat VADParams.start_secs 0.2 s Short bursts open turns. Raise to 0.3–0.4 s. Barge-in registers later by that amount.

Pin the VAD checkpoint, not just the package

The dependency your turn-taking rides on is a model file, not a version number. A voice-activity model ships as an ONNX checkpoint inside a plugin wheel; a patch release can carry a different checkpoint with different sensitivity, and your lockfile shows a version bump that looks cosmetic.

Not hypothetical. livekit/agents pull request 6816, merged 12 August 2026, replaced the bundled Silero checkpoint because issue 6815 reported the older one missing short utterances. Eleven days later issue 6956 reported the opposite failure mode on livekit-agents 1.6.10, with the user-speaking state flipping roughly 467 ms after agent speech began. The reporter ran a controlled A/B/A changing only the ONNX file — 597d30b3 against the newly bundled 1a153a22 — and the fault reproduced both times on the new checkpoint. Issue 6956 remains open as at 14 September 2026.

Why this is not a verdict: a later comment in the same thread, dated 4 September 2026, compared both checkpoints offline over digital silence and white and pink noise from −60 to −10 dBFS, found the newer checkpoint quieter on non-speech at every level tested, and concluded that leaked agent playback would open a turn under either one. The thread supports the ladder rather than the headline: the checkpoint is worth controlling, and echo is still what you rule out first. Either way, record the checkpoint hash your agent loads, not just the package version, and treat a change in that hash as a change to turn-taking behaviour needing a silent-line re-test.

Whether you can pin it differs by framework, which makes it a selection criterion alongside the others in choosing between LiveKit Agents and Pipecat. LiveKit exposes onnx_file_path on VAD.load(), documented in its own source as being there so you can use a previous version of the Silero model. Pipecat’s SileroVADAnalyzer constructor, read 14 September 2026, takes only a sample rate and a params object and loads silero_vad.onnx from its own package data, so on Pipecat the package version is the pin.

The next 7 days: what to watch upstream

If the ladder lands on step 4 there is nothing to tune; pin your framework version and track the issue instead of rewriting your session config. Statuses checked 14 September 2026, which is why the fourth row differs from a fortnight ago.

Issue Opened Status at 14 Sep 2026 What it reports
livekit/agents 6956 23 Aug 2026 Open Silero v6.2 checkpoint, false speech detection and agent self-interruption in a WebRTC session
livekit/agents 7063 31 Aug 2026 Open A confirmed false interruption leaves a stale speech-start anchor on the next user turn, 12.151 s early in one production timeline
livekit/agents 7198 9 Sep 2026 Open, labelled bug Interim and preflight transcripts do not clear the false-interruption timer, so the agent resumes and the partial transcript is lost
livekit/agents 7057 31 Aug 2026 Closed 4 Sep 2026 by PR 7061 A backchannel during agent speech made the agent’s own playback count as a user pause, pushing the learned dynamic endpointing delay to its maximum
pipecat-ai/pipecat 5683 10 Sep 2026 Open A final transcript queued behind the frame that starts a user turn is flushed by that turn’s own interruption, so the turn opens empty

When to tune this yourself, and when a threshold cannot fix it

Do it yourself when one configuration can serve every call on the route. Below roughly a hundred calls a week there is nothing to measure statistically anyway: run the ladder, change one dial, listen to twenty recordings, repeat.

The cost is not the first tuning pass. It is that every dependency bump reopens the question, and a checkpoint hash can change inside a patch release without a changelog line, so the silent-line test becomes a release gate you now own — on every route, language and campaign with a different acoustic profile.

One configuration stops being enough the moment a route mixes caller populations: a threshold set for a quiet office handset is wrong for a speakerphone in a car, and a gate tuned for English backchannels fires differently where acknowledgement tokens are longer. Zian AI runs its Customer Support Agent across 30+ languages for that reason, and supports private model deployment on customer infrastructure — which here is what makes the checkpoint yours to pin rather than your vendor’s to bump. If a threshold cannot serve your call mix, the answer is per-route configuration or a different detector, not a braver number.

Frequently asked questions

Why does my AI voice agent stop mid-sentence when nobody spoke?

Something in the inbound audio crossed a speech threshold while the agent was talking. On LiveKit Agents 1.8.1 the stock gate is min_duration 0.5 seconds and min_words 0, so half a second of anything scoring as speech will stop the agent. The usual sources are echo of the audio the agent is playing, line noise, or a one-word backchannel.

How do I prove the agent is hearing its own output?

Put the agent on a call with nobody on the far end and let it speak for sixty seconds. No human is present, so any speech event or committed transcript came from the line or from the agent. Mute the agent output and repeat. If the events stop, the cause is echo. If they continue, the cause is the line.

Is the Silero VAD checkpoint version really a production variable?

It can be. LiveKit pull request 6816, merged 12 August 2026, replaced the bundled checkpoint with the v6.2 model described in the Silero VAD v6.2 release notes, which lists higher stability on edge cases including lower quality phone calls. Issue 6956, open as at 14 September 2026, reports the opposite failure mode, and a later comment in that thread could not reproduce extra noise sensitivity from the checkpoint alone.

Should I turn interruptions off to stop this?

Not on live calls. Disabling interruptions means the caller cannot stop the agent at all, which callers experience as worse than an occasional stumble. Raise min_words to 2 or 3 first, then min_duration, then fix the audio path. LiveKit keeps interruptions enabled by default and suppresses them while echo cancellation warms up.

Does this happen more on phone calls than on web calls?

The echo cause does, and the framework defaults reflect it. The LiveKit AgentSession source, read on 14 September 2026, sets an echo-cancellation warmup of 3.0 seconds by default and sets it to None for outbound SIP calls unless you pass a value yourself, so on an outbound phone call that protection is off unless you ask for it.

Getting turn-taking right on real campaigns

Self-interruption is not a prompt problem and a better voice will not fix it. It is an audio-path problem with a four-step test, published defaults and a model file you should be pinning. If you would rather have the audio path, thresholds and checkpoint managed per route than own that release gate, Zian AI is in partnership-application beta. Apply For Partnership. More answers of this kind sit on the Zian AI FAQ hub for autonomous AI sales agents.


Related Blogs

Related from Zian AI