Accented English is not the bottleneck vendors imply. WildASR (Boson AI, arXiv, 26 March 2026) measured 2.2–6.8% word error rate on accented English across seven systems — Australian speakers were 12.1% of that accent set — against the 4.7% error rate human annotators scored on the same benchmark. Proper nouns are the real failure: AssemblyAI’s own benchmark reports a 19.6% missed entity rate on phone numbers.
Two questions get merged in most vendor demos. “Will it understand my Australian customers?” is a speech recognition (ASR) question, measured in word error rate. “Will it sound Australian?” is a text-to-speech (TTS) question, measured by nothing in particular. The second gets marketed. The first books the wrong suburb.
ASR and TTS answer different questions
A voice agent is a pipeline: speech-to-text, then a language model, then text-to-speech. An Australian-sounding TTS voice tells you nothing about whether the recogniser heard “Woolloomooloo” or “Caulfield” correctly. You can run a perfect Australian voice on top of a recogniser that mangles every second street name, and the demo will sound excellent.
The failure is silent. Nothing announces “I guessed at that suburb”. The customer says Caulfield, the transcript says Coalfield, the agent confirms the appointment, and no one finds out until the technician drives to the wrong side of Melbourne. Same class of problem as the gap between meetings booked and meetings held: the metric that looks fine is not the metric that matters. Judge TTS by listening. Judge ASR by reading transcripts of your own calls against what was actually said.
Where the “30-50% WER for accents” figure comes from
The most repeated number in this space is that accented speech runs 30–50% word error rate against 2–8% for native speakers. It is traceable, and tracing it is instructive.
The source is an eight-page undergraduate research paper in the EAAI track at AAAI-24 (pages 23327–23334), “Allot?” Is “A Lot!” Towards Developing More Generalized Speech Recognition System for Accessible Communication by Bandodkar, Agarwal, Sughosh, Singh and Choi. Its abstract states: “Notably, we found that accented speech exhibits significantly higher word error rates of 30-50%, in contrast to native speakers’ 2-8% (Baevski et al. 2020).”
The accented figure is for Indic-accented English, measured with Wav2Vec 2.0 on YouTube-crawled NPTEL lecture audio and the Svarah and IIT Madras datasets. It was never an Australian number. The “2-8%” baseline is a citation to Baevski et al. 2020, the wav2vec 2.0 paper, whose abstract reports “1.8/3.3 WER on the clean/other test sets” with full supervision and “4.8/8.2 WER” with ten minutes of labelled data — all on LibriSpeech, which is read audiobook narration. The comparison sets accented lecture audio against audiobooks read in quiet rooms, using different corpora. The paper’s own three datasets scored 51.64%, 34.31% and 4.60% WER — its headline 30–50% range does not contain its own third result.
Hold the corpus and the model set constant and the gap shrinks. WildASR, in Back to Basics: Revisiting ASR in the Age of Voice Agents (Boson AI, March 2026), put seven systems over one accented-English set and got 2.2–6.8% WER, straddling the 4.7% error rate human annotators scored on the same benchmark. Apply the rule below to that number too: the accent samples are read speech drawn from the GLOBE_V2 corpus, and the paper labels them second-language accents, so treat 2.2–6.8% as a floor rather than a forecast for phone audio. Checked 2 September 2026.
The rule this establishes: a WER number without its corpus and recording conditions is meaningless. Read audiobooks, webinar audio, meeting recordings and 8 kHz mobile calls are four different problems. A vendor figure that does not name its test set is a marketing number.
Why headline WER understates a failed booking
Word error rate weights every token identically. A missed “um” costs the same as a missed surname. On a booking call that is backwards, because proper nouns and digit strings carry the entire payload.
A 12% WER spread across filler words, false starts and backchannels can still produce a correct booking. A 5% WER concentrated in the suburb, the surname and the mobile number produces a booking wrong in every field that matters. AssemblyAI makes this point in Word error rate is broken (14 April 2026) and publishes a missed entity rate alongside WER. On its own benchmark of real-world customer audio, Universal-3 Pro missed 13.1% of names, 19.6% of phone numbers and 34.3% of emails and URLs — a vendor’s best case, on audio it chose. Roughly one phone number in five. Treat that as the shape of the problem, not a figure to quote at your own volumes.
Is proper-noun error the single biggest driver of accent-related task failure? That is widely asserted and we could not find a primary study isolating it. What is established: WildASR shows aggregate WER obscures which factors actually drive failures, and vendors now publish entity-level metrics because WER did not predict production behaviour. Treat “proper nouns dominate” as a well-motivated hypothesis to verify on your own calls.
What actually breaks on an Australian call
| Failure mode | Why Australia is harder | What to test |
|---|---|---|
| Suburb and locality names | The ABS counts 15,353 Suburbs and Localities (ASGS Edition 3; Edition 4 landed 22 July 2026 but its suburb boundaries are not due until October 2026). Many are Aboriginal-language derived and spelling does not predict pronunciation (Woolloomooloo, Coolangatta). Names repeat across states. | Your top 100 suburbs by call volume, read aloud by Australian speakers, scored as exact matches. |
| Street and place names | The Composite Gazetteer of Australia, compiled by Geoscience Australia from every state and territory naming authority, is the national register of approved place names. Its own feature service returned 289,560 records on 2 September 2026. | Full addresses spoken naturally, not dictated. Score the whole address, not the words in it. |
| Business and product names | Local brand nouns are absent from general training data. Franchise and trade names compound it. | Your own product and competitor names. Check whether keyterm or word-boost lists exist, and whether they help. |
| Spelled-out surnames | Letter confusion pairs (B/V/P, D/T, M/N) are acoustically fragile, and non-rhotic Australian English makes the letter R a live problem. | Twenty surnames spelled letter by letter, including migrant-origin surnames from your actual customer base. |
| Alphanumerics | Four-digit postcodes, ten-digit mobiles starting 04, and the 11-digit ABN. Australians group digits verbally: “double four”, “triple oh”. | Numbers read at speed, with doubles and triples. One wrong digit is a total failure, not a partial one. |
| Code-switching | The 2021 Census: “In 2021, 5.6 million people or 22 per cent, reported using a language other than English at home.” 2026 Census data does not land until June 2027. | Calls where a name or place is pronounced in the speaker’s first language mid-sentence. See multilingual AI sales agents. |
| Telephony audio itself | Narrowband codecs and mobile handover degrade the signal before the recogniser sees it, independent of accent. | Recordings pulled from your carrier, never laptop microphone audio. |
What speech-to-text vendors publish, and what they don’t
Checked on each vendor’s own pages, 2 September 2026. “Not published” means the vendor does not state it on the page linked — not a judgement about the model.
| Vendor | Own headline English WER | Test datasets named | Australian or accent-stratified figure | Entity metric |
|---|---|---|---|---|
| AssemblyAI | Universal-3.5 Pro, 4.35% pre-recorded | Buckets only for WER — synthetic medical, Accented English (India), general speech, webinar; underlying corpora unnamed. Named elsewhere for multilingual (Common Voice, FLEURS, VoxPopuli) and diarisation (DiPCo, CALLHOME, NOTSOFAR, AMI) | Accented English (India) only; no Australian breakdown | Yes — missed entity rate |
| Deepgram | Relative only: “According to Deepgram’s internal benchmarks, Nova-3 achieves up to a 36% lower WER than OpenAI Whisper on select datasets” | Not published (“select datasets”) | Not published | Not published |
| ElevenLabs | No number: “Scribe v2 achieves the lowest word error rate recorded on industry-standard benchmarks” | FLEURS shown in a chart; no figures in text | Not published | Not published |
| Google Cloud Speech-to-Text | Not published — states a single accuracy score across all customers and use cases is impractical | Not applicable; directs you to your own audio | Not published | Not published |
| Speechmatics | Not published on the benchmarking page (methodology only) | Not published on that page | Not published | Not published |
The pattern is the point. None of them publishes an Australian-English figure, and four of the five publish no absolute WER at all on the pages above. Google’s documentation tells you to compute WER yourself against audio that is acoustically representative of your use case. AssemblyAI says it plainer: “Always run your own evaluation on your actual audio data before making a vendor decision.” So any voice agent vendor claiming “we handle Australian accents” is making a claim neither they nor their upstream provider has published evidence for. Not dishonesty — an unmeasured claim you can measure.
How to run an Australian accent test in an afternoon
- Pull 30 real call recordings from your existing phone system, not scripted reads. Include mobile calls, a noisy one, and at least five where the customer has a non-English first language.
- Write the ground truth by hand. One person transcribes each call verbatim — about two hours for thirty short calls, and the part that makes the exercise valid.
- List the entities per call: suburb, street, surname, mobile number, postcode, appointment day and time, product name. Usually five to ten.
- Run all 30 through each candidate. Same audio, no tuning on the first pass. Repeat with each vendor’s keyterm list loaded with your suburbs and product names, and record both scores.
- Score two numbers. WER across the transcript, and entity accuracy: what percentage of your listed entities came through exactly right. An entity is right or wrong, never partly right.
- Score task success separately. For each call, would the resulting booking have been correct? That number will be lower than both the others, and it is the one operations cares about.
- Add an adversarial set: fifty of your hardest suburb names, twenty spelled surnames, twenty mobile numbers, read by three different Australian speakers. Twenty minutes to record, and it finds more than the thirty real calls do.
Then ask the vendor to reproduce your results. One that will run your audio and hand back raw transcripts is worth more than one with a better published benchmark. Ask which speech-to-text model sits underneath, whether you can pin a version, and what happens when the upstream provider deprecates it.
If your volume is low and your service area is two suburbs, you probably do not need this. It matters when your address surface is wide — field services, multi-site clinics, national inbound — or when a wrong booking costs a truck roll. Two pieces of Australian groundwork pair with it: the right kind of Australian phone number on the agent, and why STIR/SHAKEN does not apply here the way it does in the US. Accuracy does not help if the call is not answered.
Frequently asked questions
Is a 95-98% accuracy claim wrong?
Not wrong, but conditional. AssemblyAI’s How accurate is speech-to-text in 2026? (8 July 2026) states: “Speech-to-text accuracy tops out around 95-98% word accuracy on clean audio for the best models in 2026, but that headline number hides most of the real story.” The same page puts phone conversations at 80-88% and heavily accented speech at 75-90%. The figure applies to clean audio, and phone calls are not clean audio.
Do AI voice agents struggle with Australian accents specifically?
Less than folklore suggests. The WildASR benchmark (Boson AI, March 2026), in which Australian speakers made up 12.1% of the accented English set, found 2.2-6.8% word error rate across seven systems against the 4.7% error rate human annotators scored on the same benchmark. Those samples are read speech, not phone audio, so treat the range as a floor. The Australian difficulty is concentrated in place names and spelled-out words, not the accent as a whole.
What is a good word error rate for a booking agent?
There is no threshold worth quoting, because WER does not measure booking correctness. Measure entity accuracy on suburbs, surnames and phone numbers, then measure whether the resulting booking was right. A 5% WER concentrated in proper nouns is worse for you than a 12% WER spread over filler words.
Will training on Australian data fix it?
Partly, and only for what the data covers. Mozilla’s Common Voice v24 en-AU subset, published 21 January 2026 under CC0, has 55,673 clips — but it is elicited (read) speech, not phone conversation. Fine-tuning improves acoustic modelling of Australian vowels. It does not teach a model your suburb list or your product names. Keyterm boosting does that.
Should I ask the vendor for their word error rate?
Ask, but do not decide on it. Ask instead for the underlying speech-to-text provider and model version, whether keyterm boosting is available, and whether they will run your thirty recordings and return the raw transcripts. Google Cloud’s own accuracy documentation tells buyers to measure against acoustically representative audio of their own rather than rely on a published score.
How many recordings do I actually need?
Thirty real calls plus a ninety-item adversarial list of suburbs, spelled surnames and mobile numbers is enough to separate candidates. It is not enough for a statistical claim and should not be presented as one. It is enough to see which vendor gets Woolloomooloo right.
Where this fits
Zian AI has been running outbound acquisition since 2017: 50,769+ qualified sales appointments set. An appointment only counts when the address, the name and the number on it are right, which is why accent and entity accuracy get tested against real Australian call audio rather than a vendor benchmark.
Apply For Partnership to see how the agents handle your call recordings.