Implement text analysis solutions 4.2 Implement speech solutions

A car maker's voice assistant, written in Java, must keep recognizing spoken commands…

Study guide skill: Integrate speech as an agent modality, including custom speech models

Question B5-017

A car maker's voice assistant, written in Java, must keep recognizing spoken commands when the car is out of network coverage and use cloud speech when it's connected. Which approach fits?

Select one answer.

  • A The speech to text REST API for short audio, with responses cached on the device
  • B Embedded speech with the Speech SDK for Python and HybridSpeechConfig
  • C Embedded speech in the Speech SDK with hybrid speech (HybridSpeechConfig), after the company is approved for embedded speech's limited access ✓ Correct answer
  • D The Speech CLI installed on the car's computer, running recognition offline

Answer: C

Embedded speech is designed for on-device speech to text and text to speech when cloud connectivity is intermittent or unavailable, such as in a car, and hybrid speech combines cloud and offline recognition in the C#, C++, or Java SDK. Microsoft limits access to embedded speech, so you apply through the limited access review first. The Speech CLI and REST APIs don't support embedded speech, and the Python and Go SDKs don't support hybrid speech.

Deep dive

How it works

Embedded speech is built for on-device speech to text and text to speech when cloud connectivity is intermittent or unavailable. The docs' own examples include a car that might travel out of range. It runs inside the Speech SDK on the device, so recognition keeps working without a network.

The "use cloud when connected" part is hybrid speech. With the HybridSpeechConfig object, the app uses the cloud speech service by default and falls back to embedded speech when connectivity is limited or slow. For recognition, it switches to embedded after repeated failed attempts to reach the cloud, and it can go back to the cloud when the connection returns.

Three constraints shape the answer:

  1. Limited access: Microsoft limits access to embedded speech, so the company has to apply through the limited access review first.
  2. SDK support: Only the C#, C++, Java, Python and Go SDKs support embedded speech. The other SDKs, the Speech CLI and the REST APIs don't.
  3. Hybrid support: The Python and Go SDKs don't support HybridSpeechConfig. For hybrid, use C#, C++ or Java. A Java assistant fits.

Plan for the device too. Embedded recognition only supports mono 16-bit, 8-kHz or 16-kHz PCM WAV audio, and memory use is roughly the model's file size plus 200 MB for recognition.

The other options, and when you'd use them

  • The REST API for short audio with cached responses: REST APIs don't support embedded speech, and cached responses can't recognize new commands. Short-audio REST also returns only final results, not partial ones.
  • Embedded speech with the Python SDK and HybridSpeechConfig: Python supports embedded speech but not hybrid speech, and the app is written in Java anyway.
  • The Speech CLI running offline: The docs say the Speech CLI doesn't support embedded speech.

Exam tip

"Offline on the device" means embedded speech (after limited access approval). "Cloud first, offline fallback" means HybridSpeechConfig, which only C#, C++ and Java support.

References

Last reviewed:

Ready to test yourself? The quiz asks this and every other AI-103 question, with instant feedback. Practice this in a quiz →

All Text analysis questions · AI-103 study guide

Thanks for your feedback!