Voice AI latency is the gap between the moment a caller stops speaking and the moment the AI agent’s reply becomes audible. Human conversation sets a demanding baseline: across ten languages, the most common turn-to-turn gap was 0 ms and the cross-language mean was about +208 ms (Stivers et al., PNAS, 2009). Listener research on simulated phone calls found that perceived willingness to help starts dropping once a silence passes about 600 ms, with a statistically significant drop between 700 and 800 ms (Roberts & Francis, JASA Express Letters, 2013). In practice, a voice agent that regularly takes more than a second to respond is not just slow — it is sending a social signal the caller reads as reluctance, confusion, or a robot.

Why milliseconds decide sales calls

When teams compare AI voice agents, latency tends to come up last — often only after the first batch of real calls, when prospects start talking over the agent or hanging up mid-silence. That ordering is backwards: on an outbound call you have seconds of goodwill, and every awkward pause spends some of it.

This is a different problem from speed to lead, which is about how fast you dial a new enquiry (minutes matter there). Latency is about what happens inside the conversation, turn by turn, in milliseconds — and it is unique to voice. An email SDR can take thirty seconds to draft a reply and nobody notices; a voice agent gets a few hundred milliseconds before the silence itself becomes the message.

The human baseline: how fast people actually take turns

The most widely cited measurement of human turn-taking comes from Stivers and colleagues, published in PNAS in 2009. The team timed responses to questions in naturally occurring conversation across ten languages from five continents. The headline findings:

• The most common (modal) gap between turns was 0 ms overall, with each language’s mode falling between 0 and +200 ms.
• The cross-language mean response offset was +208 ms, with every language’s mean falling within roughly 250 ms either side of that — from Japanese at +7 ms to Danish at +469 ms.
• All ten languages showed the same pattern: speakers avoid overlapping talk and minimise silence between turns.

People manage this because they predict the end of the other person’s turn and plan a reply while still listening — something most AI pipelines only partially do. And because near-zero gaps are the norm humans are calibrated to, delays that look tiny on a dashboard are perceptually enormous on a call.

Silence is not neutral: what long gaps signal

A slow response does not read as “the system is processing”. It reads as social information. Roberts and Francis (JASA Express Letters, 2013) played 380 listeners simulated phone conversations in which someone responded affirmatively to a request — the words were identical, only the silence before the reply varied, in 100 ms steps from 200 to 1,200 ms. Ratings of the responder’s willingness to help were consistently high below 500 ms, began to drop after 600 ms, and stepped down significantly between 700 and 800 ms. Beyond about 900 ms, ratings hit a floor. (The authors caution the exact threshold was measured with American English speakers and shouldn’t be treated as a universal constant.)

On a sales call the consequences compound:

• Hang-ups. A cold-call recipient owes you nothing; dead air early in the call is the cheapest moment to bail out.
• Talk-over. Humans treat silence past a few hundred milliseconds as their cue to speak. When the agent’s delayed reply then lands, both parties collide and the call rhythm breaks.
• The “robot tell”. A consistent one-second pause before every reply is one of the fastest ways callers identify an automated system — changing their behaviour for the rest of the call.

Where the delay actually comes from: the pipeline

A typical AI voice agent chains four stages on every turn. Each contributes latency, and vendors quote whichever slice flatters them — so it pays to know the anatomy.

1. Endpointing and speech recognition (ASR)

First the system must decide the caller has finished speaking. This “endpointing” decision is a genuine trade-off: wait too little and the agent barges in on people merely pausing mid-sentence; wait too long and every turn inherits that delay. Streaming ASR transcribes as the caller talks, but the end-of-turn decision itself adds its own delay to every single turn.

2. LLM time-to-first-token

The transcript goes to a language model, and what matters is not the total generation time but the time to first token — how quickly the model starts producing a reply that can be spoken. Bigger, smarter models generally take longer to start; long prompts, retrieved knowledge-base context, and tool calls (say, a CRM lookup mid-call) all push this out further.

3. Text-to-speech time-to-first-byte

The reply is streamed into a TTS engine. Again the metric that matters is time-to-first-audio-byte, not the time to synthesise the whole sentence. Modern streaming TTS engines are fast, but voice quality, voice cloning, and language coverage all interact with speed.

4. Telephony transport

Everything above rides on top of the phone network. ITU-T Recommendation G.114 — the long-standing standard on one-way transmission time — says most applications are not significantly affected if one-way delay stays below 150 ms, recommends that 400 ms one-way not be exceeded for general network planning, and calls delays above 400 ms unacceptable for planning purposes. For AI agents that means the network alone can consume a meaningful chunk of the budget before the AI has done any thinking at all — and media routing that hairpins across continents stacks straight onto every turn.

What the delay feels like: latency bands

There is no single independently verified study that measures every layer of every platform, so treat any cross-vendor comparison as approximate. The bands below combine the peer-reviewed perception research with figures published by Telnyx — itself a voice AI vendor, so read its numbers as a competitor’s own comparison, not neutral truth.

Response gap (end of caller speech → agent audio)	How callers experience it	Source of the band
Under ~500 ms	Within the range of ordinary human turn-taking; perceived willingness stays consistently high	Stivers et al. (PNAS 2009); Roberts & Francis (JASA-EL 2013)
~600–800 ms	Noticeably a pause; listener judgements of the speaker begin to drop, with a significant step down between 700 and 800 ms	Roberts & Francis (JASA-EL 2013)
~800 ms–1.5 s	“Noticeably delayed” per Telnyx’s own comparison page; callers start filling the silence or repeating themselves	Telnyx (vendor self-published)
Above ~1.5 s	Conversation “feels broken” per Telnyx’s own comparison page; talk-over and hang-ups become likely	Telnyx (vendor self-published)

Why the average is the wrong number

Most latency claims are medians (p50) measured under favourable conditions. The number that decides whether prospects hang up is closer to p95 — your worst one-in-twenty turns — because a ten-minute call contains dozens of turns, so nearly every call includes several worst-case ones. The long tail has causes the median never sees: turns that trigger a knowledge-base or CRM lookup, long conversation context late in a call, model cold starts, network congestion.

The third-party production-call tests cited on Telnyx’s comparison page illustrate the gap: across platforms measured on 500 production calls each, p95 ran roughly 240–330 ms above the medians. A vendor quoting only an average is telling you about their best turns, not your callers’ worst ones.

The latency–quality trade-off is real — but it’s a tuning knob

Any agent can be made faster with a smaller model, shorter prompts, and no mid-call lookups — and dumber with it: worse objection handling, weaker qualification logic. Serious platforms treat this as an engineering budget: route simple turns (“yes”, “what’s this about?”) to fast paths, reserve slower reasoning for turns that need it, and stream everything so speech starts before thinking finishes.

Interruption handling (barge-in) is the other half of conversational feel. When the caller talks over the agent, does it stop within a syllable or two — and understand what it missed? An agent with brilliant fast responses that bulldozes through interruptions still feels robotic. Test both together; this is a core part of what separates good agents from bad ones in any voice agent comparison.

How to evaluate a vendor honestly

• Measure on real calls, not the web demo. Browser demos skip the telephone network entirely. Call the agent on a real phone number, from the regions your prospects are in, and time end-of-your-speech to start-of-its-speech.
• Ask for p95, and how it was measured. A trustworthy answer names the measurement span, network conditions, and tail. “Sub-X ms” with no methodology is marketing.
• Test with your own scripts and integrations. Latency balloons when the agent consults your knowledge base or CRM mid-turn; a ten-line demo prompt tells you nothing about production.
• Test interruptions deliberately. Talk over the agent, pause mid-sentence, trail off. Endpointing and barge-in behaviour show up in the first minute of adversarial testing.
• Check where the stack runs relative to your callers. Compute on another continent adds transport delay to every turn — the same physics behind G.114’s one-way delay limits.

The same discipline applies whether you’re comparing platforms head-to-head or deciding between AI voice agents and a traditional call centre: humans set the conversational bar, and any system you deploy is measured against it on every turn.

Where Zian fits

Zian AI is an autonomous AI sales agents platform that runs live phone conversations as one channel in multi-channel outreach alongside SMS, email, and WhatsApp, in 30+ languages with voice cloning. Zian does not publish a latency figure — hold us to exactly the standard above: test on real calls, with your scripts, and time the turns yourself. SmartReach AI™ orchestrates which channel and moment each prospect gets, so voice is used where a live conversation earns its keep, and for teams with data-residency constraints Zian supports private model deployment on your own infrastructure.

FAQ

What is a good response latency for an AI voice agent?

There is no single agreed number, but the human benchmark is strict: across ten languages, the most common turn gap was 0 ms and the cross-language mean was about +208 ms (Stivers et al., PNAS, 2009). Perception research on phone-call silences found listener judgements start declining past about 600 ms and drop significantly between 700 and 800 ms. A practical reading: under ~500 ms feels conversational, and consistently exceeding a second costs you calls.

Does the phone network’s delay count towards voice AI latency?

Yes, and it’s often forgotten. ITU-T Recommendation G.114, the standard on one-way transmission time, says most applications are unaffected below 150 ms one-way and recommends not exceeding 400 ms for network planning. That network budget is consumed before the AI pipeline — speech recognition, language model, and speech synthesis — adds its own delay on top.

Why does the vendor’s demo feel faster than real calls?

Web demos usually run over a direct internet connection, skip carrier telephony, use short prompts with no CRM or knowledge-base lookups, and are hosted close to the demo audience. Production calls add carrier hops, your real script and data integrations, and long-context turns late in calls. Always benchmark by phoning the agent on a real number with your own use case configured.

What’s the difference between p50 and p95 latency, and which should I ask for?

p50 (median) is the midpoint turn; p95 is the latency your slowest one-in-twenty turns exceed. Because a single call contains dozens of turns, almost every call will experience p95-level delays several times. Ask vendors for p95 measured from end-of-caller-speech to start-of-agent-audio on production phone calls — and ask what conditions the measurement used.

Can a voice agent respond too quickly?

Yes. If the endpointing is too aggressive, the agent treats a caller’s mid-sentence pause as the end of their turn and barges in — which is as damaging as slow responses. Humans also occasionally take longer on turns that warrant thought, so a well-tuned agent varies its timing rather than firing back instantly on every turn. Evaluate interruption handling and endpointing together with raw speed.

Does Zian publish a latency figure?

No. Zian doesn’t publish a headline latency number, because a single number without methodology is exactly the kind of claim this article warns against. If you evaluate Zian — or any platform — do it on live phone calls with your own scripts, regions, and integrations, and measure the turn gaps yourself.

Want autonomous AI sales agents working the phone, SMS, email, and WhatsApp?
Zian AI will partner with a limited number of teams during its waitlist beta.
Apply For Partnership