Pepys

Blog

Transcription accuracy by domain: why the same AI model breaks on specialized speech

General speech recognition is close to solved on clean read audio. The moment jargon, drug names and case citations enter, the same model's error rate can jump two to ten times. Here is the spread, by domain, with the data.

By Pepys ·

The short version

General AI transcription hits 2.7% word error rate on clean read speech (OpenAI Whisper paper, Radford et al., 2022), but the same class of model makes roughly twice as many errors as a human once audio leaves that distribution. In specialized fields the gap is quantified: clinical-entity error runs 4 to 51 percent worse than the same recording's general word error rate.

The number on the box is the clean-read number

General speech recognition is close to solved on one specific kind of audio. The best zero-shot Whisper scores 2.7% word error rate on LibriSpeech test-clean, near the roughly 5.8% a human makes on the same set (OpenAI Whisper paper, Radford et al., 2022). That is the number that tends to end up on a marketing page.

Move the microphone into a real workplace and the same class of model makes roughly twice as many errors as a human. Whisper's own headline gain came from a 55.2% average relative error reduction across out-of-distribution datasets, which is another way of saying every dataset outside clean read speech was a problem worth fixing.

Specialized vocabulary is exactly that out-of-distribution case. Accuracy is not one number. It is a number per domain, and the spread between those numbers is the whole story. If you want the general picture first, we covered how accurate AI transcription is and what word error rate actually measures separately.

The jargon penalty, quantified

A study of speech recognition on medical named entities put a figure on the thing everyone suspects: error on the medical terms themselves ran 4 to 51 percent worse than the general word error rate for the same audio (Olatunji et al., 2024). The overall transcript can look fine while the words that matter quietly rot.

Whisper-large is a clean example. On the same recordings it went from 37.3% general word error rate to 45.4% on medical entities. The model did not get worse at English. It got worse at the handful of words a clinician would actually read back.

This is why a domain matters more than a brand. The gap is not vendor A versus vendor B. It is clean-speech error versus specialized-vocabulary error, and it opens up inside a single model on a single file.

Medicine: an order-of-magnitude spread

A systematic review of 29 studies of AI speech recognition for clinical documentation found word error rates from 8.7% in controlled dictation to over 50% in conversational, multi-speaker clinical settings. Same task, same category of model, an order of magnitude apart depending on how real the speech was.

Dictation is the easy case: one trained speaker, deliberately, into a microphone. A ward round with overlapping voices and drug names is the hard case, and it is the one that matters. Crosstalk compounds the problem in every field, which we broke out in how well AI handles crosstalk.

The words you most need are the ones it drops

Averages hide where the errors land. The same medical-entity study found that even the strongest general model recalled only 42% of medication names and 33% of protected health information (Olatunji et al., 2024). Most drug names and identifiers were missed or mangled while the overall word error rate still looked respectable.

And a general model does not only mishear jargon. It can invent it. An external audit of Whisper found whole hallucinated phrases in about 1% of transcriptions (187 of 13,140 segments, 1.4%), and 38% of those hallucinations contained explicit harms: invented violence, made-up names, false authority (Koenecke et al., FAccT 2024).

A fabricated sentence in a medical or legal record is a different category of problem from a misspelling. It reads as fact, in the model's confident register, with nothing to flag that it was never said.

Therapy: the error lands on the critical line

Mental-health audio shows the pattern at its sharpest. In a HIPAA-compliant test on real psychotherapy sessions, a commercial system averaged 25% word error rate (range 8 to 74%), but climbed to 34% on the clinician-flagged, harm-related sentences (Miner et al., 2020).

Error was highest on exactly the lines a clinician cannot afford to lose. That is the general shape of domain error, not an exception: the specialized, high-stakes token is the one the model is least sure of, because it is the one that appeared least often in training.

Radiology and the tax on dense terminology

You might expect a model tuned for a specialist domain to erase the gap. It narrows it, but the tax remains. A Whisper Large-v2 adapted for French radiology reached 17.121% word error rate on radiological reports, more than double the same base model's 7.67% error rate on general Common Voice audio (PMC, 2024).

Radiology reports are dense with technical and numeric terminology, and numbers and rare terms are precisely what general training under-samples. Even after adaptation, dense-vocabulary speech costs you. The lesson generalizes: a domain-specific model is a better starting point, not a finished transcript.

Legal: the bar sits above the output

The problem is not only that specialized error rates are high. It is that specialized domains demand an accuracy general speech recognition does not reach. UK Crown Court transcripts must be delivered at 99.5% accuracy, and the US NCRA's Registered Professional Reporter skills test uses a 95% threshold (figures collected by Speechmatics). Last verified: 2026-07-13.

Set those against raw transcription of names, case citations and legal terms and the distance is plain. This is the reason certified legal work still runs through a reviewer, and why the file you get from a model is a draft to check rather than a record to file. We cover the workflow in legal transcription.

What actually closes the gap

The same research that measures the gap also measures the cure, and it is not buy a better general model. It is give the model the domain.

Fine-tuning on clinical speech cut medical-entity word error rate by 48% relative in the medical-entity study (Whisper-medium, Olatunji et al., 2024). A lighter approach, biasing the model toward a list of expected names and terms, recovered up to 17.8 to 20.0% relative word error rate on entity-heavy audio through biasing lists and prompting (contextual-biasing study, 2023). Both point the same direction: a domain or correction pass beats raw output.

The practical version for one recording is smaller but real: feed the system the names, acronyms and terms it will encounter, then clean the transcript against them afterward. We wrote up the hands-on steps in how to improve transcription accuracy. If you want the head-to-head model numbers, they live in our AI transcription accuracy benchmark.

Where this leaves you

The honest summary is a spread, not a headline. On clean read speech, AI transcription is close to human. On specialized vocabulary, the drug names, the case citations, the radiological terms, the harm-related line in a therapy session, the same model can be two to ten times worse, and its worst errors cluster on the words you most need.

So the question is never whether AI transcription is accurate. It is accurate on what. Pick a current, large speech model, then plan for a review pass on the domain terms. The review is where the accuracy standard your field actually requires gets met.

That is roughly the shape Pepys is built around: a current model for the first pass, real exports so the transcript lands in whatever tool you correct it in, and no length cap so a two-hour deposition or a full research interview goes through in one piece. It will not read a garbled drug name for you. Nothing will. But it gives you a clean, editable first draft and the file to fix it in, which for specialized audio is the honest job. You can start with the audio-to-text tool or see pricing.

Questions, answered

Which domain is hardest for AI transcription?

Conversational, multi-speaker specialized speech, such as clinical ward rounds, therapy sessions and courtrooms. A 29-study review of clinical documentation found word error rates from 8.7% in controlled dictation to over 50% in conversational clinical settings, and psychotherapy sessions averaged 25% word error rate with harm-related lines at 34%.

Does a medical or legal transcription tool fix the jargon problem?

It narrows the gap but does not erase it. Fine-tuning on clinical speech cut medical-entity word error rate by 48% relative in one study, yet a domain-adapted radiology model still landed at about 17% word error rate against 7.67% on general audio. A specialist model plus a review pass, not either alone, is what reaches field standards.

Will a general model like Whisper get medical or legal terms right?

Often not the ones that matter. On medical audio the strongest general model recalled only 42% of medication names and 33% of protected health information, and general Whisper can even hallucinate whole phrases in about 1% of clips. Overall word error rate can look fine while the critical tokens are wrong.

How do I improve accuracy on domain-specific vocabulary?

Give the model the domain. Biasing toward a list of expected names, acronyms and terms recovered up to about 20% relative word error rate on entity-heavy audio in published work. In practice: supply your term list up front and correct the draft against it. See our guide on how to improve transcription accuracy.

Is AI transcription accurate enough for court or medical records on its own?

No, not as the final record. UK Crown Court transcripts require 99.5% accuracy and the NCRA skills test uses 95%; raw output on names and citations falls short, and hallucinations make an unreviewed transcript risky as an official record. Use it for the first draft, then review.

References

  1. 1.OpenAI Whisper paper (Radford et al., 2022)PMLR (ICML 2023 proceedings)
  2. 2.Systematic review: AI speech recognition for clinical documentation (29 studies)npj Digital Medicine / PMC (2025)
  3. 3.Performant ASR Models for Medical Entities in Accented Speech (Olatunji et al., 2024)arXiv (Interspeech 2024)
  4. 4.Careless Whisper: Speech-to-Text Hallucination Harms (Koenecke et al., FAccT 2024)ACM FAccT 2024 / arXiv
  5. 5.Assessing the accuracy of automatic speech recognition for psychotherapy (Miner et al., 2020)npj Digital Medicine / PMC
  6. 6.French-language ASR for radiology (domain-adapted Whisper Large-v2)PMC (2024)
  7. 7.What word error rate is acceptable for legal transcriptionSpeechmatics
  8. 8.Contextual Biasing of Named-Entities with Large Language Models (2023)arXiv

Keep reading

Don't just take our word for it.

Ask ChatGPT, Claude, or Perplexity what Pepys is and who it's for. One click, and your favorite AI does the homework.

Built for the four complaints above

Any length, pay once, real exports, no training on your audio. Start free with 60 minutes, no card.