How accurate is AI transcription? The benchmark numbers, read honestly.

Ask "how accurate is AI transcription" and you tend to get a single number back, usually a very good one. Published benchmarks show the best models transcribing clean read English at around 2.7% word error rate. That is real. It is also nearly useless as a guide to how a tool will handle your two-hour conference call.

Because the same class of model that scores 2.7% on clean speech scores above 30% on far-field group conversation - a room recorded on distant microphones with people talking over each other. That is not a different model failing. It is the same benchmark family, moved from an audiobook to a real room. The honest answer to the accuracy question is a range, not a point, and the audio decides where in that range you land.

Last verified: 2026-07-12. Every number below comes from a published paper, dataset, or benchmark, cited in the sources. Pepys has run none of these benchmarks itself; this is a reading of the public record, not a first-party test.

Almost every accuracy claim in speech recognition is a word error rate, or WER. The definition is mechanical: WER = (Substitutions + Deletions + Insertions) / N, where N is the number of words in the reference transcript. Align the machine's output against a human reference, count the words it swapped, dropped, and added, then divide by the total.

The intuition that makes it readable: a 5% WER means roughly one word in twenty is wrong, and a 2.7% WER is about one word in thirty-seven. Lower is better, and the scale does not feel linear - the gap between 2% and 5% is the gap between a transcript you skim-correct and one you fight. We unpack the formula, its edge cases, and why it flatters clean audio in word error rate explained.

WER has blind spots worth naming up front. It weighs a dropped "the" the same as a mangled proper noun, and it says nothing about punctuation, speaker labels, or whether the one wrong word was the one that mattered. It is the best single number we have, which is not the same as a complete one.

Start with the good news, because it is genuinely good. In the paper that introduced Whisper, OpenAI's large model transcribes LibriSpeech test-clean at 2.7% WER. LibriSpeech is the canonical clean-speech benchmark: roughly 1,000 hours of read English audiobook narration from the public-domain LibriVox project, split into an easier test-clean and a harder test-other.

Test-clean is, by now, nearly solved. Public leaderboards suggest the very best systems have pushed it toward 2% WER or below, which means the clean set no longer separates the top models from each other - they are all bunched against the same wall. When a vendor quotes you a sub-3% accuracy figure, this - clean, read, single-speaker English - is almost always the condition it came from.

Part of why modern models generalize at all is scale. Whisper was trained on 680,000 hours of multilingual, weakly-supervised audio - not curated studio recordings but the messy web. That is the reason it holds up on audio it never saw in training, and the reason the clean-speech number, impressive as it is, undersells the more interesting part.

The headline you have seen - "AI now matches human transcribers" - traces back to a specific, careful experiment, and it is worth reading in the original. In 2016 Microsoft measured professional human transcribers on the NIST 2000 telephone-speech test set and found they were not perfect: 5.9% WER on the structured Switchboard portion and 11.3% on the casual CallHome portion. Its system that year matched them at 5.8% and 11.0%.

A later, more rigorous study using multiple independent transcribers put the human error rate at 5.1% WER on Switchboard; Microsoft's 2017 system reached 5.1% as well. That 5.1% is the closest thing we have to a true professional-human floor on conversational telephone speech - and it is emphatically not zero. Humans mishear, guess, and drop words too.

This is the number that matters most for reading every "human parity" claim honestly. Parity was demonstrated on a particular benchmark - telephone conversations - not declared universally. The right sentence is "the model matched humans on this task," never "the model is more accurate than humans." On the mixed real-world Kincaid46 set (news, podcasts, meetings, phone calls), the Whisper paper compares its best model against five professional transcription services and reports it landing within about a percentage point of them - the best computer-assisted service beat it by 1.15 points, and pure-human transcription by only a fraction of a point. Close to human-level, in the paper's words, but not past it.

Now the part vendors leave off the slide. The same benchmarks that produce the flattering clean numbers also measure the fall, and the fall is steep.

Move from test-clean to test-other - still read audiobook speech, just noisier and more accented - and Whisper's large model goes from 2.7% to 5.6% WER. Roughly double, for what is still a cooperative single speaker. Move from structured to casual conversation and the same thing happens to humans: 5.9% on Switchboard, 11.3% on CallHome, for the same expert transcribers. Speaking style alone can double the error rate.

Then move into a real room. On the CHiME-6 challenge - a dinner party recorded on distant microphones, with overlapping speakers and background noise - the best system reached only about 30% WER, five to six times the clean-speech rate. And that was an improvement on the prior CHiME-5 round, where the best result was 46.1%. Far-field, multi-speaker audio is where accuracy goes to die, and it is exactly the audio most people actually have. If your recording carries more than one voice, the fixes are upstream: see how to improve transcription accuracy and, for crosstalk specifically, how to transcribe multiple speakers.

Every figure so far is English. Change the language or the accent and the picture shifts again, usually for the worse, and usually unevenly.

Accent-diverse speech is tested with crowdsourced corpora like Mozilla Common Voice, which now spans more than 250 languages of volunteer-read speech (a July 2020 snapshot held 7,226 hours across 54 languages). The point of a corpus like that is precisely to surface the accents on which WER climbs well above the clean-English numbers. If you work with strong accents, transcribe accented English covers what actually helps.

Across languages the unevenness is stark. Google's FLEURS benchmark covers 102 languages with a small amount of speech each - on the order of a dozen hours per language - and error rates rise sharply for lower-resource languages. A single "AI transcription accuracy" number, in other words, is really a claim about high-resource languages like English. For everything else, the honest answer is: it varies, often a lot.

The load-bearing lesson from all of this is not any single figure. It is that a low benchmark score does not, by itself, mean real-world robustness. The two can diverge, and the Whisper paper contains a clean demonstration of it.

Models trained directly on LibriSpeech can post excellent LibriSpeech scores and still fall apart elsewhere. Whisper, which was not trained to chase that leaderboard, makes 55.2% fewer errors on average across other, out-of-distribution datasets than models optimized for LibriSpeech. A tool can win the benchmark and lose the room. When you evaluate a transcription tool, the number that matters is its mean WER across noisy, telephony, and meeting audio - not its best score on the cleanest set.

So read every accuracy claim with three questions. What audio was it measured on (clean read speech, or something like yours)? What model version produced it (Whisper's 5.6% test-other is the original 2022 large model; later versions report slightly different numbers)? And is it a mean across hard conditions, or a best case on an easy one? The practical, non-numeric companion to this piece is how accurate is AI transcription, which answers the same question from the user's chair.

Stack the benchmarks and the honest summary is short. On clean, read, single-speaker English, the best AI models sit at roughly 2.7% WER and are closing on 2% - at or just past the ~5% floor of professional human transcribers on conversational speech. That is a genuine milestone. On accented, noisy, multi-speaker, far-field audio, the same models run several times worse, past 30% WER in the hardest rooms. Both facts are true at once, and any answer that gives you only one of them is selling something.

Pepys runs a Whisper-class model, so these are the numbers that apply to it too. We did not run our own benchmark, and we are not going to quote one we did not run. For clean single-speaker English, expect the low-single-digit WER the published benchmarks show. For messy multi-speaker or far-field recordings, expect materially worse, and plan a correction pass. That is the honest fit, and it is the same one every tool in this class actually delivers - the difference is whether they tell you. For the practical, tool-by-tool version, we wrote up what real users hit in 100 Reddit threads about transcription.