Which languages AI transcription struggles with – and why

People ask "how accurate is AI transcription?" as if there were one figure. There isn't. On a single widely cited open benchmark, the same model swings roughly 40x depending only on the language you feed it – single-digit error on English and Spanish, triple-digit error on Amharic and Sindhi.

The numbers below come from primary research: the OpenAI Whisper paper, an Interspeech Arabic-dialect study, a Data in Brief code-switching corpus, and a ServiceNow benchmark. They describe specific model versions (mostly Whisper large-v2), not "all AI transcription" – newer systems have closed some of the worst gaps. But the shape of the curve is stable, and it is the thing worth understanding before you trust a tool with a recording in a language other than English. If you want the ground rules for reading these figures, start with how accurate AI transcription actually is and what word error rate means.

Last verified: 2026-07-12.

Start with the easy end. On Google's FLEURS benchmark, Whisper large-v2 posts a word error rate (WER) in the low single digits for the well-resourced European languages: Spanish 3.0%, Italian 4.0%, Portuguese 4.3%, English 4.2%, German 4.5%. French sits a little higher at 8.3%. This is the near-human tier – the accuracy most English speakers assume is universal because it is what they experience.

WER is simply the share of words the model gets wrong (substitutions, deletions, insertions) against a human reference. A 4% WER means roughly one word in twenty-five needs a fix. For clean audio in these languages, that is genuinely useful out of the box, and a light correction pass gets you the rest of the way.

Hold this baseline in mind, because every hard case below is defined by how far it falls from it.

At the other end, the model doesn't just get worse – it collapses. On FLEURS, Whisper large-v2 records WERs above 100% for a cluster of low-resource languages: Sindhi 156.5%, Amharic 140.3%, Shona 121.0%, Myanmar 115.7%, Mongolian 110.5%, Assamese 106.2%, Bengali 104.1%, Somali 102.9%, Punjabi 102.4%, Lao 101.5%, Malayalam 100.7%.

A WER over 100% is not a typo, and it does not mean "more than every word is wrong." WER counts insertions as well as substitutions and deletions, so when a model loses the thread and hallucinates – repeating phrases, inventing words, padding the output – it can emit more wrong words than the reference contained. The score blows past 100% because the denominator is the reference length, not the model's output length. In practice, a triple-digit WER means the transcript is unusable, not merely rough.

Between the two extremes sits an awkward middle band: tonal and non-Latin-script languages that are workable but clearly harder. On the same benchmark Whisper large-v2 records Mandarin Chinese 14.7%, Korean 14.3%, Arabic 16.0%, then climbing steeply through Nepali 47.1%, Pashto 93.7%, Telugu 99.0% and Khmer 99.7% right up to the edge of the cliff.

The cause isn't mysterious, and it isn't really about the languages themselves. It's about data. Whisper was trained on 680,000 hours of audio, but the mix is lopsided: about 65% (438,218 hours) is English speech recognition, another 18% is translation, and only around 17% (117,113 hours) is multilingual speech recognition spread across dozens of languages.

Inside that multilingual slice the imbalance repeats. The best-served languages get tens of thousands of hours – Chinese around 23,000 – while the weakest get less than a single hour of training audio. A model that has heard under an hour of a language has essentially no chance of transcribing it reliably. The cliff in the accuracy table is a direct shadow of the cliff in the training data.

This is also why speaker count doesn't predict accuracy. Hindi has hundreds of millions of speakers, yet on Common Voice 9 Whisper large-v2 scores 21.9% WER for Hindi against 9.4% for English and 5.6% for Spanish (and Bengali, another huge language, lands at 103.4%). Popularity in the world is not the same as representation in the training set. If you're trying to raise accuracy on a language the model half-knows, the upstream moves in how to improve transcription accuracy matter more than the tool you pick.

Low-resource status is one cliff. Dialect is a separate one, and it catches languages that look well-supported on paper. A model can be fluent in a written standard and still fail on the spoken variety people actually use.

Arabic is the clearest case. Zero-shot, Whisper-large scores about 10.36% WER on Modern Standard Arabic (the formal register of news and writing) but 53.82% on Moroccan Darija – roughly five times worse for the same language family. Egyptian Arabic, the most-represented dialect, still triples the standard error at 31.39%. And on dialects the model has essentially never seen, it falls off the same cliff as any low-resource language: on YouTube audio, Whisper large-v2 records Algerian 103.44%, UAE 102.83% and Yemeni 102.66% WER, with Jordanian 72.80% and Palestinian 51.92% not far behind.

Swiss German shows the trap from a different angle. It has no standard orthography, so there is no "correct" written form to transcribe to. Whisper's response is to quietly change the task: instead of transcribing the dialect verbatim, it translates the speech into Standard German. The research concludes the model is usable "so long as the Standard German output is desired" – which is a polite way of saying it is not really transcribing at all. That is the difference between transcription and translation, and it's worth knowing which one your tool is silently doing.

Plenty of the world doesn't speak one language at a time. Hinglish (Hindi and English braided together), Spanglish, and dozens of other blends switch languages inside a single sentence – and code-switching is its own failure mode. Published work finds it raises WER 30-50% relative to monolingual speech, and it is not a niche problem: India alone has over 250 million code-switchers. On a dedicated Hinglish corpus, Whisper-medium posts a 16% WER for adults and 18% for children – workable, but a clear step down from clean monolingual audio.

A 2025 frontier-ASR benchmark from ServiceNow found Whisper took the largest code-switching penalty of the systems tested, for a revealing reason: it tends to translate bilingual audio into English rather than transcribe it verbatim. Across code-switched language pairs, Whisper Large V3 Turbo ranged from 0.16 to 0.61 WER, and its degradation relative to clean English peaked at +0.85 on German-English, while newer systems such as ElevenLabs Scribe V2, Gemini 3 Flash and AssemblyAI Universal-3 Pro showed the smallest deltas. (That benchmark is a vendor lab's own code-switch set, so read it as directional, not gospel.)

If your recordings are bilingual by nature, the practical move is to expect an edit pass and, where you can, keep speakers in one language per segment. Related: our notes on transcribing accented English, which is a milder cousin of the same problem.

Some languages get hit twice – low training data and features the model handles badly. Cantonese is the textbook example: it is tonal (linguists count six to nine contrastive tones depending on how you cut it), and it is data-scarce next to Mandarin.

The result is steep. On Common Voice Cantonese, the smallest Whisper model (Whisper-tiny) posts a 49.5% character error rate before any tuning – roughly half the characters wrong. Fine-tuning with LoRA cut that to 11.1%, which tells you two useful things: the base performance on tonal low-resource languages is poor, and it is fixable with targeted training you, as a user, don't control. Larger Whisper models do better than tiny out of the box, but Cantonese stays firmly in the hard column.

It's tempting to read these tables as the final word. They aren't. FLEURS – the benchmark behind most of the numbers above – covers 102 languages with only about 12 hours of evaluation speech each. For a low-resource language, a handful of hard clips can move the headline WER a lot, and the same data scarcity that hobbles the models also thins the tests that grade them.

The numbers are also version-specific. These are chiefly Whisper large-v2, the flagship of the original paper. Later Whisper releases and non-Whisper systems have improved several low-resource and code-switch scores. Treat any single figure as "a widely cited open benchmark shows," not "AI transcription is exactly this good." The direction of the curve – excellent on high-resource, worse on low-resource, worst on unseen dialects and code-switching – is what generalizes.

Pepys uses the same class of large multilingual models these benchmarks measure, so we won't pretend the physics is different for us. Accuracy tracks the published curve: excellent on high-resource languages, weaker on low-resource, dialectal and code-switched audio. Our marketing surface ships in 15 locales, but that's the interface, not a promise that every language transcribes equally well.

The honest guidance: for a supported high-resource language and clean audio, expect near-human accuracy and a light edit. For a heavy dialect or a mixed-language recording, expect to edit more, and where you can, record in the standard variety – it measurably helps. If you need the output in a different language than the audio, that's a translation step, and our audio-to-text tool, transcript translator and guide to translating a transcript keep the two jobs separate so you can see what the model actually heard first.

And the genuinely honest part: for a very low-resource language sitting in the triple-digit-WER tier, no general cloud model is the right answer today. A specialized local model, a community-built fine-tune, or a human transcriber will beat it. We'd rather tell you that than sell you a transcript you can't use. If you want the field notes behind this whole topic, we also read 100 Reddit threads about transcription – accents and crosstalk came up again and again.