AI Voice Agent Voicemail Detection, Diagnosed - Zian AI

AI Voice Agent Voicemail Detection, Diagnosed

Voicemail detection misfires in five documented ways, and one field on the call record tells you which. Twilio’s AMD cuts human from machine at 2,400 ms of speech, LiveKit’s at 2,500 ms — both inside the 1,800–3,000 ms band Twilio’s FAQ says business greetings typically occupy. Rule out that overlap before you touch a toggle.

Answering machine detection (AMD) makes an irreversible decision in the first four seconds, on incomplete audio. It fails two ways: a false human, where a mailbox is treated as a person and your agent talks to a beep, or a false machine, where a person is treated as a mailbox and gets read a script. They look identical on the dashboard; the fixes are opposites.

Each cause below carries the test that separates it from the others, using fields Twilio, LiveKit, Retell, Vapi and Bland already publish. We hold no cross-platform incidence data and could not locate a published dataset, so the order is by how many callers are exposed on default settings. Dead air mid-conversation is a different fault: see why an AI voice agent goes silent on tool calls and the reason-code Rosetta table.

First, read the one field that already holds the verdict

Get the verdict string and its timing before you form a theory: timing is what separates a threshold problem from a plumbing problem. Twilio publishes a second field almost nobody uses, MachineDetectionDuration — “Time in milliseconds that AMD took to reach a verdict”.

Platform Field to pull Value that means voicemail What it does not tell you
Twilio Programmable Voice AnsweredBy plus MachineDetectionDuration machine_start, or machine_end_beep / machine_end_silence / machine_end_other Whether a human verdict was correct. Use the duration test below.
LiveKit Agents AMD result.category, result.transcript machine-vm, or machine-unavailable for a full or unconfigured mailbox uncertain is not a machine — LiveKit’s docs say “Treat as a human and proceed with normal conversation.”
Retell AI disconnection_reason voicemail_reached An IVR is a separate verdict, ivr_reached. See cause 3.
Vapi Ended Reason, plus the detection provider set on the assistant voicemail Which of its four providers ran — their behaviour differs.
Bland The voicemail object on the call Whichever action actually ran, which may not be the one you set See cause 4: a message silently overrides hangup.

Cause 1: the 1,800/3,000 overlap band

Twilio’s AMD FAQ and best practices publishes a greeting-length profile nobody joins up to the threshold defaults. Personal human greetings “are typically pretty short and less than 1800 ms”; business greetings “~1800-3000 ms”; machine greetings “typically longer, >3000 ms”.

Now look at where the engines cut. Twilio’s MachineDetectionSpeechThreshold defaults to 2,400 ms, defined in its AMD guide as the measuring stick: “Durations lower than this value may be interpreted as a human, longer as a machine.” LiveKit’s AMD documentation sets human_speech_threshold to 2.5 seconds — “Maximum length of a ‘short greeting.’ Shorter speech takes the fast-path human heuristic; longer speech is treated as machine-like and defers to the LLM.”

Two independently built engines landed 100 ms apart, both inside the band Twilio itself says business humans typically occupy. Twilio does not leave the consequence to inference: the same FAQ publishes a different threshold setting for each segment. For “individuals, residences, or mobile phones” it recommends “setting MachineDetectionSpeechThreshold to 1800-2000 ms”; for businesses, “somewhere between 1800 and 3000”. Call it the 1,800/3,000 rule: the shipped 2,400 ms default sits above Twilio’s own recommended range for consumer numbers and inside its range for business numbers, so one global threshold is a trade-off between your two segments, not a value that can be right for both. Twilio states the reverse error outright — “The AMD engine may, for example, interpret a very short two-second voicemail greeting as a human picking up.”

Answerer Published greeting length Against a 2,400–2,500 ms cut Dominant error What to change
Personal mobile or residence < 1,800 ms Human, comfortably False human on short mailbox greetings Raise MachineDetectionSpeechEndThreshold toward 2,500 ms
Business switchboard ~1,800–3,000 ms Straddles the cut False machine — a live receptionist gets a voicemail script Raise MachineDetectionSpeechThreshold; on LiveKit raise human_speech_threshold above 2.5 s
Answering machine or mailbox > 3,000 ms Machine, comfortably Rarely misread on length alone Nothing; this is the easy case
Mixed mobile and business list Both bands at once One cut, two different recommended ranges Both errors at once Split the campaign — Twilio publishes a different setting for each segment.

The test. Pull every call with AnsweredBy=human and plot MachineDetectionDuration. A spike under about 3,000 ms, on calls where no caller speech follows your agent’s opening line, is false humans. If the durations instead bunch at your MachineDetectionTimeout, go to cause 2.

Cause 2: the detector’s clock ran during ringback, not after answer

The tell is a verdict arriving before anyone picked up. livekit/agents #6187, filed 22 June 2026 and closed as completed on 5 August 2026, described AMD’s detection_timeout being armed at audio-track subscription. With carrier early media the track subscribes during ringback, so the 20-second budget burned down while the phone still rang — the reporter’s timeline shows it expiring at +26 s and the mailbox answering at +32 s. The fix that shipped is PR #6580, merged on 5 August 2026: “Start the budget only when listening begins, and settle immediately if the participant disappears before audio arrives.” A community pull request, #6386, had also proposed a separate 60-second sip_answer_timeout guard; it was closed unmerged on 28 July 2026, so do not go looking for that option — it is not in the shipped library. LiveKit’s current docs now state the behaviour plainly for no_speech_threshold: “The clock starts when the call is answered, so ringback and early media don’t count against it.”

The mirror-image request is still open. Issue #6895, filed 18 August 2026 and open as at 12 September 2026, asks for an opt-in to process carrier early-media audio, because some carriers deliver the voicemail greeting over SIP 183 Session Progress before any 200 OK. On those routes, deferring to the answered state “discards the only audio that was ever going to be available”.

The test. Put the SIP answer timestamp beside the verdict timestamp. A verdict predating the answer is a clock problem and no threshold tuning will touch it. A verdict after answer with zero speech duration is the early-media variant: the greeting played before answer and your detector never heard it.

Cause 3: it was classified as an IVR, so voicemail handling never fired

“Detection is on and it still does not hang up” has a documented cause that is not the voicemail switch at all: the verdict landed in a different bucket, with its own switch. Retell’s voicemail and IVR documentation says so directly: “This most likely means the call is being classified as an IVR rather than a voicemail. Because the two detection types are handled separately, voicemail hang up will not trigger in this case.” Retell adds two scope limits: detection “will only run for the first 3 minutes of the call”, and never on inbound calls.

Bland has the same split. Its call recipes documentation states that with ivr_mode: true, any voicemail action resolves to ignore: “A phone menu has no voicemail, so IVR mode suppresses voicemail handling entirely. The legacy amd alias sets the same switch.” An inherited config still setting amd has voicemail handling switched off.

The test. Read the terminal reason, not the behaviour. ivr_reached on Retell or machine-ivr on LiveKit means the audio was classified as a menu, so fix the IVR switch. Navigating that menu is a separate layer: see why DTMF digits go missing on AI voice agents.

Cause 4: the verdict was right and the handler could not act on it

The verdict lands, the log line prints, the call runs on. pipecat-ai/pipecat #5330, filed 14 August 2026 and closed as completed on 4 September 2026, reported that the on_voicemail_detected handler in Pipecat’s own docs could not end the call: the frame it pushes was discarded inside the detector’s closed gate, so “the pipeline runs on until some other mechanism tears it down”. On Bland the trap is in the request body — “supplying a message implies leaving one”, so {"action": "hangup", "message": "..."} leaves the message instead of hanging up.

livekit/agents #6996, filed 26 August 2026 and closed as completed on 1 September 2026 by PR #7026, documented a stream desync in which a live human answering with a quick “hello” got no fast verdict and the detector dragged to its timeout. The reporter published 14 days of their own production traffic: 123 AMD verdicts, eight stalling at roughly 17–20 seconds, and five of 36 confirmed-human verdicts (14%) returning on the slow path.

The test. Measure the gap between verdict and action. A verdict at 2 s and a hang-up at 40 s is a handler problem; a verdict at 18 s on a call answered at 1 s is the desync. Check your framework version against the dates above first — three of the four framework defects cited here were closed between 5 August and 4 September 2026, so a forum post written in the present tense may describe a fixed bug.

Cause 5: the handset answered, not the person

This is the honest limit of the current state of the art, and it is widening. Apple’s iPhone User Guide describes Call Screening: “Call Screening automatically answers calls from unknown numbers without interrupting you. Once the caller shares their name and reason for their call, your iPhone rings and shares their response so you can decide if you want to pick up.”

Read that as a telephony event: the call is answered, a synthesised voice speaks and waits for a reply. Not a human, not a mailbox, no beep. Neither Twilio’s published AnsweredBy values (machine_start, human, fax, unknown, plus three machine_end_* variants) nor LiveKit’s five AMD categories (human, machine-ivr, machine-vm, machine-unavailable, uncertain) contain a category for it, as published on those pages on 12 September 2026.

Australia has a version that is easy to miss. Telstra’s MessageBank and ring time page states that Apple Live Voicemail “is Apple’s on-device voicemail service”, that “It’s turned on by default from iOS 18 onwards”, that with it “the ring time is 15 seconds”, and that “To extend ring time beyond 15 seconds on Apple devices, you must turn Live Voicemail off”. A default-on, on-device mailbox answering at 15 seconds is a different target from the US carrier mailbox most tuning advice assumes, and Telstra’s ring-time options of 15, 20, 25 or 30 seconds are the subscriber’s choice, not yours.

The test. There is no field for this one. Listen to three recordings where AnsweredBy=human but the transcript shows one short prompt then silence. A synthesised voice asking who is calling and why is screening. Treat it as unknown, leave nothing, retry.

How long should I let AMD listen? The derivation

Derive the budget rather than guess it. With Twilio defaults, time to verdict is the greeting duration plus MachineDetectionSpeechEndThreshold — “The number of milliseconds of silence after speech activity at which point the speech activity is considered complete” — default 1,200 ms. So a personal mobile human resolves in up to 1,800 + 1,200 = 3.0 s, a business human in 3.0–4.2 s, a machine past 4.2 s. Twilio’s own published average is consistent with that derivation: “On average Twilio’s AMD will return results within ~4 seconds after the call was answered with default optional configuration values.” Treat those 4 seconds as a cross-check on your own MachineDetectionDuration distribution, not as a target.

Now price the fix. To stop short mailbox greetings reading as human, Twilio’s guide describes raising MachineDetectionSpeechEndThreshold to about 2,500 ms, because “For short voicemails, there is typically 1000-2000 ms of audio followed by 1200-2400 ms of silence and then additional audio before the beep.” It also states the bill: “a change of 1200 ms to 2500 ms increases human detection delay by 1300ms.” That 1.3 seconds is paid on every human answer, on top of their 3.0–4.2 s; the benefit lands only on short-greeting mailboxes. Per 100 answered calls, where H is the number of those calls a human answers, the added silence totals 1.3 × H seconds:

Human answers per 100 connected Added dead air across those 100 calls
40 52 seconds
60 78 seconds
80 104 seconds

Ceilings differ sharply. Twilio’s MachineDetectionTimeout defaults to 30 seconds and only bites in DetectMessageEnd mode. LiveKit’s timeout defaults to 20 seconds, but its docs warn the timer “resets once the participant’s audio track is subscribed, so the effective ceiling can reach roughly twice this value”, and that with the default wait_until_finished=True “A long greeting can therefore run past timeout“. Vapi’s beepMaxAwaitSeconds defaults to 30 seconds, and its voicemail detection page notes “Most voicemail systems play 10-20 seconds of greeting before the beep”. Retell runs detection for the first three minutes and says it “generally adds under 30ms of latency”.

Two scope limits close a surprising number of these tickets. Twilio’s FAQ states that “Elastic SIP trunking calls cannot utilize AMD because they bypass the Programmable Voice infrastructure” — so if your platform reaches the PSTN over a SIP trunk, what you are configuring is the framework’s own detector, with different defaults. And on LiveKit, “AMD runs once at the start of the call, on the first user utterance. It doesn’t monitor continuously”, so a transfer leg is never re-classified.

When to tune this yourself, and when it stops paying

The method above is complete and you can run it. The cost is the honest part. Twilio is blunt about the sample required — “Testing should be done with a large number of carriers, on a diverse set of devices, with varied message responses” — and warns that hyper-tuning against one handset “can result in unexpected behavior once applications are deployed widely”. That means a labelled call set, per-segment configs and a regression run at every framework upgrade — work that belongs in a pre-go-live test harness.

Your situation Tune it yourself Hand it over
One segment, one country, one carrier mix Yes — a single threshold set is defensible Not yet
Mixed mobile and business list Only if you can split the campaign per the 1,800/3,000 rule If you cannot split it, one config is always wrong for one segment
Multiple countries or carriers Rarely — Twilio states international accuracy “may be slightly reduced” Yes: per-destination configs and a labelled set per country
No labelled call set, no test harness No — you would be tuning against anecdotes Build the labelled set first, whoever runs it

Zian AI’s Outbound Appointment Setter and Appointment Show-Specialist agents place live phone calls into exactly this problem, alongside SMS, email and WhatsApp in 30+ languages, which is why detection thresholds belong per destination rather than once per account. Zian AI is currently in partnership-application beta. Apply For Partnership.

Frequently asked questions

Why is voicemailDetectionEnabled not working on my calls?

In the order this page works through them: the call reached an IVR and was classified under a separate switch; it was placed over a transport that cannot use carrier-side detection; a config field silently overrode your action; or the verdict arrived outside the window you were watching. Read the terminal reason string first.

Why does my agent leave a message when a real person answered?

That is a false machine, and on a business number it is usually the threshold overlap. Twilio’s FAQ puts business greetings at “~1800-3000 ms” while its default speech threshold cuts at 2,400 ms, so a receptionist’s full greeting can be scored machine-like. Raise MachineDetectionSpeechThreshold for business lists, and move those numbers into their own campaign.

Is Twilio’s AMD really 94% accurate?

That figure comes from Twilio’s launch announcement for the AMD system that replaced its deprecated original, which states it “offers a 94% accuracy rate across a large sample set of calls from the US and Canada”. That page carries a datePublished of 21 March 2017 in its own structured data, and Twilio’s current AMD guide and AMD FAQ, both opened on 12 September 2026, do not repeat the number. They publish a mode-specific claim instead: “MachineDetection=DetectMessageEnd accuracy is close to 100% accurate in US destinations with default settings”, with the caveat that for international dialling “accuracy may be slightly reduced”. Quote the mode and the destination, not a single number.

Does voicemail detection work on inbound calls?

No, on the platforms that document it. Retell’s voicemail and IVR page states: “Voicemail and IVR detection only run on outbound calls, where your agent dials out and may reach a voicemail or automated menu. On inbound calls a person is calling your agent, so there is no voicemail to detect and these settings have no effect.”

Am I charged for a call that went to voicemail?

Yes. Twilio’s Call resource documentation states: “A completed call indicates that a connection was established, and audio data was transferred. This can occur when a call is answered by a person, an IVR phone tree menu, or even a voicemail. Completed calls, regardless of the outcome, are charged against your project balance.” The same page adds that a call’s price “only reflects the charge for connectivity. Charges for other call-related features such as Answering Machine Detection, Text-To-Speech, and SIP REFER are not included in this value.” The full outcome-by-outcome billing matrix is in what unanswered AI calls actually cost you.

How long should AMD be allowed to listen before it gives up?

With Twilio defaults a verdict lands at greeting length plus 1,200 ms of silence, putting a human at 3.0-4.2 seconds and a machine past 4.2 seconds, matching Twilio’s published “~4 seconds” average. Extending that silence threshold to 2,500 ms catches short mailbox greetings but adds 1,300 ms to every human detection. Set the ceiling from the destination: Telstra’s mobile ring-time options are 15, 20, 25 or 30 seconds.

Related Blogs

Related from Zian AI