A clinic's speech-to-text results miss medical terms, and recordings are made in a noisy…

How it works

Out of the box, speech recognition uses a base model trained on Microsoft-owned data that reflects commonly used spoken language. It works well in most scenarios. Custom speech doesn't replace it: a custom model augments the base model. There are two main levers:

• Text data improves recognition of domain-specific vocabulary, such as medical terms.
• Audio data with reference transcriptions improves recognition in the app's specific audio conditions, such as a noisy room.

You can also train with structured text for patterned data, specify custom pronunciations, and customize display formatting. The workflow is to upload test data, train, test (the service reports a word error rate) and deploy. If you train with audio data, choose a resource in a region with dedicated hardware for audio training.

How you use the model depends on the API. Except for batch transcription, you must deploy a custom endpoint to use a custom speech model. With the Batch transcription API, a hosted endpoint isn't required, which saves resources if the model is only used for batch. Custom models work with real-time speech to text, speech translation and batch transcription. You pay for model usage and endpoint hosting.

The other options, and when you'd use them

• Audio data can't be used to train custom models: Audio with reference transcriptions is exactly how you adapt to noisy conditions.
• Custom speech replaces the base model: A custom model augments the base model, and you choose a base model as a starting point.
• Only available through the Realtime API: Custom models can be used for real-time speech to text, speech translation and batch transcription.

Exam tip

"Jargon" means text data, and "noise or accents" means audio plus transcripts. "Custom model without an endpoint" means batch transcription.

References

• What is custom speech?
• What is batch transcription?