Size a voice agent pool by concurrent sessions, not by CPU. At 240 calls in the peak hour with a 4.5 minute average handle time you are offering 18 erlangs of traffic, and Erlang B at 1% blocking needs 28 concurrent slots — three workers at LiveKit’s documented 10 jobs per 4-core agent server, four once a rolling deploy is allowed for.
How many workers do I need so calls don’t get rejected at peak?
Capacity for voice agents is a worker-count problem that presents as a CPU problem. A web request holds a worker for 200 milliseconds; a voice session holds one for the whole conversation. So a pool can sit at 60% CPU, look healthy on every graph you own, and still refuse the next inbound call.
The frameworks make this explicit. In LiveKit Agents the AgentServer constructor exposes load_fnc and load_threshold; the published default load_fnc is average CPU utilisation over a 5-second window and the default load_threshold is 0.7, above which the server stops accepting jobs. Pipecat Cloud inverts it: its scaling docs state that each instance runs one active session at a time, so concurrency is the instance count and a pool at its max-agents cap returns HTTP 429. Two mechanisms, one consequence: plan concurrent sessions, and use CPU only as a sanity check.
The worked calculation, from calls per hour to worker count
Step 1 — offered traffic, in erlangs. ITU-T Recommendation E.600 defines traffic intensity as the product of arrival rate and mean holding time, in erlangs; one erlang is the traffic intensity in a pool of resources when just one resource is busy.
A = calls per hour × average handle time in hours
A = 240 × (4.5 / 60) = 18 erlangs
Step 2 — erlangs to slots, using Erlang B. Erlang B is the classical loss formula: calls finding every slot busy are rejected rather than queued, which is what a full worker pool does. ITU-T Recommendation E.520 recommends reading circuit counts from tables based on the classical Erlang B formula at a loss probability of 1% during the mean busy hour for automatic operation. At A = 18, 27 slots blocks 1.11% of calls and 28 blocks 0.71%. So 28 slots.
Step 3 — slots to workers. LiveKit’s self-hosted guidance recommends 4 cores and 8 GB per agent server, handling 10 to 25 concurrent jobs depending on the components in use. Plan at the bottom of that band until you measure your own pipeline: 28 ÷ 10 = 2.8 → 3 workers.
Step 4 — headroom for ramp, not for average. Pipecat Cloud’s capacity-planning guide publishes a formula worth borrowing whatever you run: Optimal Reserved = MAX(Baseline Sessions, CPS × Idle Creation Delay), CPS being calls per second during the burst, their idle creation delay about 30 seconds. Substitute your own start time: at 1 call per second with a 45-second pod start, MAX(28, 45) = 45 slots, or 5 workers.
Step 5 — headroom for deployment. A draining worker stops accepting jobs but keeps its live calls; LiveKit’s drain_timeout defaults to one hour, and its guidance suggests a 10+ minute grace period for voice apps. During a rolling deploy your accepting capacity is N − 1, so add one. Final: 4 workers steady, 6 with the burst in scope.
Substituting a second set of inputs
Now 60 calls in the peak hour at a 2 minute average handle time. A = 60 × (2 / 60) = 2 erlangs. Erlang B: 6 slots blocks 1.21%, 7 slots blocks 0.34%, so 7 slots, and 7 ÷ 10 = 0.7 → 1 worker.
Here the arithmetic lies to you: one worker is one rolling deploy, one OOM kill or one bad load sample from zero capacity, so the floor is two. The ratio is the lesson — 18 erlangs needed 28 slots, a multiple of 1.56; 2 erlangs needed 7 slots, a multiple of 3.5. Small pools need proportionally far more headroom than large ones.
The occupancy gap: why CPU looks calm while calls are refused
Call this the occupancy gap: at a 1% blocking target a correctly sized pool runs well below full occupancy at its own peak, and the smaller the pool, the lower that figure. Moderate utilisation at the moment calls are refused is the expected state, not a contradiction. From Erlang B at 1% blocking, calls per hour shown at 4.5 minute handle time:
| Offered traffic | Calls per hour | Slots at 1% blocking | Slots per erlang | Peak occupancy |
|---|---|---|---|---|
| 2 erlangs | 27 | 7 | 3.50 | 28.5% |
| 18 erlangs | 240 | 28 | 1.56 | 63.8% |
| 30 erlangs | 400 | 42 | 1.40 | 70.9% |
| 100 erlangs | 1,333 | 117 | 1.17 | 84.6% |
A large pool can run at 85% and still block only 1%; a two-erlang pool driven past 30% starts refusing people. Autoscale a small deployment on a 70% CPU trigger and it fires long after callers have been turned away. The carrier tier in front of this one has a separate ceiling, covered in scaling AI calling from 100 to 10,000 calls.
Why one worker holds fewer concurrent calls than you expect
The accept threshold bites before the machine does. LiveKit published a load test of a voice-to-voice app: 30 agents in 30 rooms with 30 simulated participants publishing looping speech, running Silero VAD, on one 4-core 8 GB machine, peaking at about 3.8 cores and 2.8 GB. That is roughly 0.127 cores per session by our own arithmetic on their figures, so with the default load_threshold of 0.7 the server stops accepting at 2.8 of its 4 cores — about 22 sessions. The threshold is your ceiling, not the hardware.
Idle processes are not free. LiveKit’s server options documentation states that in production the default idle process count is math.ceil(cpu_count) in Python and Math.min(os.availableParallelism(), 4) in Node.js. Issue livekit/agents #7117, opened and closed as completed on 4 September 2026, measured about 96 MB per idle job process moving from shared to private memory once each child ran a full garbage collection, which un-shares the pages holding every preloaded object. The closing pull request recorded on that issue is #7120, fix: preserve forkserver copy-on-write sharing, which shows as merged as at 14 September 2026, so check the version you are running. Four idle processes at that figure is roughly 384 MB per worker before a single call: irrelevant on 8 GB, decisive on a 1 GB container.
The load signal can simply be wrong. Issue livekit/agents #7102, opened 3 September 2026 and closed as completed on 8 September 2026, is the cleanest published example: the cgroup v2 CPU monitor read a non-monotonic whole-machine cpu.stat, so a worker reported a load of 0.80 and then −93 while the host idled at a load average of 0.02, flipping to full for about 2.5 seconds at a time. The reporter documented one lost call in 623 over three days — a SIP participant joined inside that window, the server logged no servers available, and the caller heard 39 seconds of silence. Pull request #7113 merged on 8 September 2026, but the lesson outlives it: admission control built on a utilisation reading fails in the direction of refusing work.
Symptom to instrument: the capacity threshold table
Most capacity incidents are misdiagnosed because the first instrument reached for is CPU.
| Symptom observed | What it usually means | Instrument to check | Action |
|---|---|---|---|
| Calls refused at the trunk with SIP 486, CPU flat | Dispatch or rule configuration, not capacity | The SIP log line, and the dispatch rule fields read back from the API | Fix the rule before adding a worker |
| Server logs no servers available, CPU under 20% | A worker reporting itself full when it is not | The value load_fnc returns, against the count of active jobs |
Upgrade past the sample-validation fix, or override load_fnc |
| No refusals, but answer latency climbing | Cold starts, not saturation | Warm instance count against session starts per minute | Raise prewarmed capacity |
| HTTP 429 on session start | A hard pool cap working as designed | Active sessions against the configured maximum | Raise the cap or route the overflow |
| CPU flat, memory climbing, processes killed | Per-process memory is the constraint | Private_Dirty in /proc/<pid>/smaps_rollup |
Reduce idle processes or raise the memory limit |
| Pool saturated but demand has not changed | The pool is consuming itself | Requests counted by origin, not aggregate throughput | Stop the internal caller |
What we measured when our own PHP worker pool saturated itself
That last row is not hypothetical, and the example is not a voice deployment. First-party measurement on our own WordPress publishing stack at zian.ai, 12 September 2026: the website, not an agent runtime, and no customer traffic involved. It runs on a PHP-FPM pool with pm.max_children = 4. It repeatedly returned HTTP 502 with the socket backlog climbing past 400 queued connections. We misattributed the cause twice — first to our own automation running concurrently, then to organic crawler load. Both fitted the symptom; both were wrong. The real cause was WordPress running pingback discovery on every post save, making the site fetch itself over every link in the saved post. Measured directly: one post update produced 86 loopback HTTP requests in 60 seconds, while a page view produced 0. Those loopbacks queued in the same four-worker pool as real visitors; when it filled they timed out, retried, and sustained the loop at about 44 requests a minute. That served 607 HTTP 502s in two minutes, 598 to the loopback client itself and four to real search crawlers. Suppressing the self-fetch took the loopback count from 86 to 0 and the backlog to 0, re-measured immediately.
The test that settled it took ninety seconds and it was not a CPU graph — it was counting requests by origin. A worker pool saturated by its own internal traffic looks identical to one saturated by demand, and the instrument that distinguishes them is the origin of the request. Substitute voice for PHP and the shape holds: retry storms, an agent that redials, a supervisor spawning a second session per call, all consuming slots your demand model says are free.
What to do when you cannot add workers
- Reject fast. LiveKit documents that a rejected job request is reassigned to the next available agent server, and that rejection means the server cannot handle the job, not that the job is invalid. At the carrier edge, returning busy immediately hands the caller back to your carrier plan, which can route them to a human queue or voicemail. The alternative is what #7102 recorded: 39 seconds of silence, then a hang-up. Failing in under a second beats succeeding in forty.
- Queue only against a measured wait, shorter than caller patience, with something to play. Pipecat’s guidance is a hold message for phone and a connecting state for web, because cold starts cannot be eliminated.
- Route the overflow to a second destination that is not the agent pool, decided in advance — the same design as building failover into an AI voice agent, and readable afterwards only if your call-drop reason codes distinguish a refusal from a failure.
How to load test this before peak rather than during it
Copy the shape the vendors publish. LiveKit documents an lk load-test subcommand that simulates publishers and subscribers in a room, with the warning to run it from a machine with plenty of CPU and bandwidth and ulimit -n 65535. Four things a generic load test misses:
- The worker count at which the first job is refused, not where latency degrades. Different numbers; the first loses calls.
- Load held for a full average handle time. A 30-second burst against 4.5-minute calls never reaches steady-state occupancy and flatters you badly.
- A cold start timed under load, and a rolling deploy run at peak — nobody rehearses the second, and it removes a worker from the accepting pool while the rest are full. Pipecat Cloud publishes around 10 seconds as a best-case cold start for a small image and calls it a floor rather than a promise.
The wider sequence is in how to test an AI voice agent before go-live.
Run the pool yourself, or hand the tier over
| Condition | Run the worker pool yourself | Hand the tier over |
|---|---|---|
| Under about 5 erlangs at peak | Yes — two workers, fixed count, no autoscaler | Not worth the integration |
| Traffic must stay on your own infrastructure | Yes — and you own the pager | Only with private model deployment |
The recurring cost is not the servers. It is someone who re-derives the erlang figure when handle time changes, rehearses a rolling deploy at peak, and reads load_fnc values rather than CPU graphs when the pager goes off. Where the constraint is regulatory rather than operational, private AI deployment for sales agents keeps the compute tier inside your own network. Zian AI runs agents on customer infrastructure where that is the requirement; the sizing method above applies whoever runs the pool.
Frequently asked questions
Does adding CPU fix calls being refused at peak?
Usually not. A voice job holds a worker for the whole call, so the constraint is concurrent sessions rather than cycles. At 18 erlangs and a 1 percent blocking target a pool needs 28 slots and runs about 64 percent occupied at peak, so CPU looks calm while calls are refused.
What is the default load threshold in LiveKit Agents?
The default load function is the average CPU utilisation of the agent server over a 5 second window and the default load threshold is 0.7, as published in the LiveKit server options documentation. Above that value the server stops accepting new jobs, and LiveKit states that neither parameter can be changed in LiveKit Cloud.
How many concurrent calls can one voice agent worker hold?
LiveKit recommends 4 cores and 8 GB per agent server and states that such a server handles 10 to 25 concurrent jobs depending on the components in use. Pipecat Cloud runs one active session per instance instead. Size at the bottom of any published band until you measure your own pipeline.
Should a full pool queue a call or reject it?
Reject fast if another destination exists. A caller held in silence hangs up, and a rejected job is handed to the next available agent server. Queueing helps only when the expected wait is shorter than caller patience. Overflow routing beats both.
How do I load test a voice agent pool before peak?
Copy the method the vendors publish. LiveKit ran 30 agents in 30 rooms with 30 simulated participants publishing looping speech on one 4 core machine and recorded peak usage of about 3.8 cores and 2.8 GB. Record the worker count at which the first job is refused.
Sizing a pool for your own peak
To have the compute tier behind your agents sized and operated to this standard, including private deployment on your own infrastructure, Apply For Partnership. Zian AI is currently in partnership-application beta.