When my AI agent calls a company and gets a phone tree, the call moves through six stages, and the slow one is classification, not the keypress. LiveKit’s answering-machine detection waits up to 10 seconds for any speech and its detection timeout defaults to 20 seconds, with an effective ceiling near double; Twilio’s Programmable Voice AMD defaults to 30 seconds. Digits come after that.
This is the timeline: menu detection, DTMF transport and the hold queue. It is not a fault tree – if your digits are already being sent and the IVR ignores them, that diagnosis lives in why AI voice agent DTMF digits get missed. It is also the mirror of AI voice agents versus IVR phone trees, which covers what your callers tolerate when they hit a menu. This is your agent hitting somebody else’s.
The six stages of an outbound call that lands in a phone tree
Every stage below is gated by a timer a named vendor publishes a default for. That is the useful part: almost none of this is up to your prompt.
| Stage | What has to happen to move on | The published clock that governs it |
|---|---|---|
| 1. Dial and early media | The trunk accepts the INVITE and may start audio at a 183 Session Progress with an SDP body, before anyone answers |
LiveKit’s AMD ignores pre-answer audio; an opt-in to process it is open as issue #6895 |
| 2. Answer | A final 200 OK arrives and the SIP call status becomes active |
LiveKit: the no-speech clock “starts when the call is answered, so ringback and early media don’t count against it” |
| 3. Classify: person, voicemail or menu? | The detector settles on a category. Longest stage. | LiveKit no_speech_threshold 10 s, timeout 20 s with an effective ceiling near double; Twilio MachineDetectionTimeout default 30 s (range 3-59) |
| 4. Hear the whole menu | The menu finishes its options before the agent decides | Retell AI’s Press Digit “Pause Detection Delay”, default 1000 ms, range 0-5000 ms |
| 5. Press a digit | The far end registers the tone, in a transport it accepts | Retell captures RFC 2833 by default and states no transport for the press-digit send side; Vapi documents “w” for a 0.5s pause and “W” for a 1s pause between digits |
| 6. Hold queue, then a human or a hang-up | Somebody speaks, or the agent gives up | Retell: “voicemail and IVR detection will only run for the first 3 minutes of the call” |
Stage 6 carries the constraint most teams miss. If your agent spends four minutes in a hold queue and the line then drops into a departmental voicemail, Retell’s detection window has already closed and the agent talks to the greeting. That is a consequence of a documented three-minute window, not a bug.
Why deciding “this is a phone tree” is the hardest part of the call
Recognising an IVR is the same unsolved problem as answering-machine detection, for a concrete reason: at the signalling layer a menu and a person look identical. LiveKit Agents issue #6125 (open, filed 16 June 2026) sets out the trunk-side SIP capture for a human-answered call and a carrier-voicemail call and reports them as structurally identical – “INVITE → 100 → 183 Session Progress (early media) → 200 OK” on both – concluding “there is no signaling-level way to distinguish them; the distinction only exists in the media.” That capture compares voicemail with a person rather than a menu with a person; it generalises because a phone tree also answers with a final 200 OK.
So the classifier has to listen, and listening costs seconds. LiveKit documents the behaviour: AMD “runs once at the start of the call, on the first user utterance. It doesn’t monitor continuously,” and “while AMD is running, the agent’s speech is paused so it doesn’t talk over a voicemail greeting before classification completes.” It returns one of five categories – human, machine-ivr, machine-vm, machine-unavailable, uncertain – running a fast-path heuristic and an LLM classifier in parallel, first to conclude wins.
A second timing trap precedes the classifier. LiveKit’s AMD defers audio processing until the SIP call status is active, which is right when pre-answer audio is only a ringback tone. Issue #6895 (open, filed 18 August 2026) asks for an opt-in to process carrier early media, because some carriers deliver the announcement or greeting itself in a 183 Session Progress with an SDP body. The reporter posted a capture across four of their own outbound trunks into Latin American mobile carriers, 311 call attempts: early media before the final response on 92-100% of calls per trunk, average gap between the 183 and the 200 OK of 17.6 to 23.8 seconds, maximum 47. Their own caveat travels with the table – an SDP body in the 183 proves the mechanism is active, “not that every one of these is carrying spoken content rather than a higher-quality ringback tone”. A LiveKit maintainer welcomed a pull request on 21 August 2026.
Twilio’s Programmable Voice AMD has no IVR category at all. Its documented AnsweredBy values with MachineDetection=Enable are “machine_start, human, fax, unknown”, and with DetectMessageEnd, “machine_end_beep, machine_end_silence, machine_end_other, human, fax, unknown”. No value in either set means “phone tree”, so a menu has to land on one of the others – in practice machine_start or unknown – and your agent works out the difference from the audio.
What each platform can actually do once a machine answers
Check this before promising anyone your agent can work a supplier’s menu. Every cell was read on the vendor’s own current documentation on 12 September 2026.
| Platform / SDK | IVR classified separately from voicemail? | Navigates the menu itself? | What the vendor’s own docs say |
|---|---|---|---|
| LiveKit Agents, Python | Yes – machine-ivr is one of five AMD categories |
Yes, automatically | “In Python, the session automatically starts IVR navigation when ivr_detection is enabled (the default).” |
| LiveKit Agents, Node.js | Yes, the same five categories | No | “The Node.js SDK doesn’t support IVR navigation, so the agent should handle machine-ivr the same as human and let the main agent respond.” |
| Retell AI, single or multi-prompt | Yes – a separate IVR Hangup toggle and an ivr_reached disconnection reason |
Yes, Press Digit tool | “IVR hangup is separate from IVR navigation, which lets your agent interact with and navigate through IVR menus using DTMF tones.” |
| Retell AI, conversation flow | Yes, same detection | Yes, Press Digit Node | “When in this node, the agent will not speak. Instead, it evaluates whether it should press a digit and determines which specific digit to press.” |
| Vapi | Voicemail detection is a voicemail classifier; IVR handling is prompt-led rather than a published detection class | Yes, DTMF tool plus prompting | “Vapi offers the DTMF tool to enable your assistants to navigate IVR menus and enter digits (for example, member IDs, account numbers, and dates).” |
| Twilio Programmable Voice AMD | No IVR value in the result set | Not its job – it classifies who answered, it does not navigate | AnsweredBy “results can be: machine_start, human, fax, unknown” |
The LiveKit rows catch teams out, because that is a capability asymmetry inside one vendor: same product, same AMD, same five categories, IVR navigation on one SDK and not the other. On a Node agent the vendor’s own instruction is to treat a detected menu as though a human answered – so your agent starts talking to a recording.
Can an AI voice agent press 1? Yes, if the digit survives the transport
Pressing a key is not one thing. LiveKit describes the mechanism precisely: DTMF tones “are transmitted over RTP using the telephone-event/8000 payload format, which ensures reliable signaling across codecs by sending event codes, durations, and volumes rather than the audio signal itself.” Retell states it “captures RFC 2833 DTMF (keypad tones sent as RTP events) by default.”
The far end decides whether it accepts that, and it is negotiated in the SDP at call setup – a property of your trunk, which is why it belongs with connecting an AI agent to your SIP trunk rather than with your prompt.
The third transport, SIP INFO, is where a real gap shows. LiveKit Agents issue #6329, “Allow SIP INFO messages to be sent to SIP participant”, has been open since 6 July 2026, filed by a developer whose agent sits inside a RingCentral workflow that “uses SIP INFO messages to coordinate the bot in the workflow”. A commenter replied on 3 August 2026 cross-linking a design discussion on the livekit/sip side, with a proposed surface – an lk.sip.SendInfo RPC and a new SipInfo data-packet event – and no maintainer has answered on the thread. Dated, numbered and still open: that is what a transport gap looks like when you can check it yourself.
The receiving side moved in the same period. Pipecat issue #4436 asked whether a self-hosted Pipecat agent on the LiveKit transport could receive DTMF from inbound SIP callers; the answer then was no, and it closed as completed on 23 July 2026 after a pull request added sip_dtmf_received handling. Pinned to an older release, you do not have it.
The 3/90 rule: when to stop navigating and route to a human
Navigation needs a stopping condition or a stuck agent burns a call leg until the carrier drops it. Here is the rule, with the arithmetic that produces it instead of an opinion.
The 3/90 rule. Abandon menu navigation at whichever comes first: three distinct menu levels, ninety seconds of elapsed call time without a human, or any menu heard twice – a repeat is a failure, not a level. On abandonment take the published zero-out path if the menu offered one; otherwise end the call and queue a retry rather than sitting in the queue.
Why three levels: Retell’s Press Digit Node documentation gives “Menu repeated 3 times” as its worked example of an edge-case transition, and its post-call extraction schema ships dedicated ivr_loop, ivr_steps_count and ivr_retries_count fields – the vendor expects looping to be counted and acted on. Vapi’s IVR guide independently lands on three attempts: fast, then 0.5-second gaps, then 1-second gaps, then speak the option aloud instead.
Why ninety seconds, worked end to end:
| Line item | Seconds | Where the number comes from |
|---|---|---|
| Classification ceiling before the agent may act | 40 | LiveKit AMD timeout default 20 s; the timer resets on track subscription “so the effective ceiling can reach roughly twice this value”. A ceiling only with wait_until_finished set to False – on the default a long greeting runs past it |
| Deliberate silence before each of three presses | 3 | Retell Press Digit Pause Detection Delay, default 1000 ms, x 3 levels |
| Inter-digit padding on one four-key extension entry | 3 | Vapi documents W as a 1s pause and a slow retry of 1W2W3#; a four-key entry at that spacing carries three pauses |
| Menu playback across three levels – your input, substitute your own | 44 | Your own call recordings, not a vendor default – about 15 s a level, and the only input here that does not come off a vendor page |
| Total before a human has said a word | 90 | Still inside Retell’s 3-minute detection window, with half of it left for the hold queue |
Substitute your own playback figure and the threshold moves. The point of the working is that the fixed costs – 40 seconds of classification, 3 seconds of enforced pause, 3 seconds of digit padding – are vendor defaults and cannot be prompted away. Add a three-level menu that takes about 44 seconds to read and the budget lands on ninety – half of Retell’s three-minute detection window, with the other half left for the hold queue. That is why the rule sits there rather than at a round two minutes.
Two things follow. Silence has to be explicit: Retell’s sample interaction rules tell the agent to “Wait silently or respond with NO_RESPONSE_NEEDED if prompted to hold”, and Vapi’s guide says to “Stay silent while waiting by replying with a single space character”. And when abandonment means a transfer rather than a retry, the context has to travel with it – those failure modes are in why AI voice agent transfers to a human fail.
Where the line is on navigating somebody else’s menu
Dialling a supplier, insurer or partner support line on your own behalf and working their menu to the right department is ordinary business – it is Retell’s own worked example, a clinic’s agent calling an insurer to verify coverage. Using the same machinery to get past a consumer’s call screening, or to defeat a system built to stop unsolicited calls, is not. Telemarketing and do-not-call rules apply whether a human or a model dialled, and that question is answered by the regulator in your market, not by your platform.
Who owns each figure on this page
| Figure or claim | Whose figure it is | Checked |
|---|---|---|
Five AMD categories; runs once, not continuously; no Node.js IVR navigation; no_speech_threshold 10 s; timeout 20 s, ceiling near double |
LiveKit, answering machine detection | 12 Sep 2026 |
telephone-event/8000 over RTP; ivr_detection default |
LiveKit, DTMF docs | 12 Sep 2026 |
| Pause delay 1000 ms (0-5000); “Menu repeated 3 times”; IVR extraction fields | Retell AI, Press digit and Press Digit Node | 12 Sep 2026 |
3-minute detection window; IVR hangup separate from navigation; ivr_reached |
Retell AI, Handle voicemail and IVR | 12 Sep 2026 |
| RFC 2833 by default; the SDP-versus-RTP payload-type test | Retell AI, Capture DTMF input from user | 12 Sep 2026 |
| “w” and “W” pause characters; three-step retry ladder; single-space silence | Vapi, IVR Navigation | 12 Sep 2026 |
MachineDetectionTimeout default 30 s (3-59); AnsweredBy value sets |
Twilio, Answering Machine Detection | 12 Sep 2026 |
| Issues #6125, #6329, #6895 and Pipecat #4436, states and dates | LiveKit and Pipecat public issue trackers | 12 Sep 2026 |
Frequently asked questions
Can an AI voice agent press 1 on a phone menu?
Yes, on every major platform, but through a tool call rather than as speech. Retell exposes a Press Digit tool and a Press Digit Node, Vapi exposes a DTMF tool, and LiveKit’s Python SDK starts navigation automatically on a machine-ivr result. The keys are 0-9, * and #, and a full sequence such as an extension can be sent in one press.
How long before my agent knows it has reached a menu rather than a person?
Tens of seconds, not milliseconds. LiveKit’s AMD documentation gives a default no_speech_threshold of 10 seconds and a default detection timeout of 20 seconds, and notes that because the timer resets when the audio track is subscribed, “the effective ceiling can reach roughly twice this value”. Twilio’s separate Programmable Voice AMD defaults to a 30-second MachineDetectionTimeout.
Why does my agent press the right digit and the menu ignores it?
Because the digit has to survive the transport, and the far end chooses which transports it accepts. Retell’s own documentation points at the test: inspect the call’s packet capture “for a DTMF payload-type mismatch between the SDP and the RTP packets” (Retell AI, Capture DTMF input from user, checked 12 September 2026).
Does the agent hear the whole menu before it chooses an option?
Only if you let it. Retell’s Press Digit tool has a Pause Detection Delay defaulting to 1000 ms and adjustable to 5000 ms so the agent captures the full menu first, and Vapi’s guidance is to “WAIT for all options to be spoken before proceeding”. Pressing early is the most common cause of a menu repeating itself.
What should the agent do while it is sitting in a hold queue?
Nothing audible. Retell and Vapi both document an explicit silence behaviour for this: respond with a no-response marker, or reply with a single space so nothing is spoken. Retell’s voicemail and IVR detection runs only for the first three minutes of the call, so a long queue can outlast the classifier.
Build outbound agents that know when to stop pressing
Zian AI builds autonomous AI sales agents for live phone, SMS, email and WhatsApp outreach, including an Outbound Appointment Setter and a Sales Call Closer, with SmartReach AI(TM) orchestrating message, channel and timing and PrecisionPitch AI(TM) continuously split-testing approaches. Zian AI is currently in partnership-application beta. Apply For Partnership.