Implement generative AI and agentic solutions 2.3 Optimize and operationalize generative AI systems
An app sends prompts to one model router deployment.
Study guide skill: Orchestrate multiple models, flows, or hybrid LLM and rules engines
Question B6-028
An app sends prompts to one model router deployment. The business wants the router to always pick the highest-quality model for each prompt regardless of cost, and some prompts are longer than the smallest underlying models can accept. Which two configuration choices fit? (Choose two.)
Select 2 answers.
Answer: B, E
In Quality mode the router picks the highest-quality model for the prompt and ignores cost, while Balanced (the default) and Cost pick the most cost-effective model within a quality band. The effective context window is limited by the smallest underlying model, so a model subset is the documented way to support larger contexts. You don't need to deploy the supported models separately, except Claude models.
Deep dive
How it works
Model router is a single deployment that picks an underlying model for each prompt. Two settings matter here.
Routing mode. Supported options are Quality, Cost and Balanced (the default):
- Quality picks the highest-quality rated model for the prompt, ignoring the cost.
- Balanced (the default) considers all underlying models within a small quality range (for example 1% to 2% of the best model for that prompt) and picks the most cost-effective one.
- Cost uses a larger quality band (for example 5% to 6%) and also picks the most cost-effective model.
"Always the highest quality regardless of cost" is Quality mode.
Model subset. The router's effective context window is limited by the smallest underlying model. A call with a larger context succeeds only if the prompt happens to be routed to a model that can take it. For larger contexts, use a model subset to select only the models that support your requirements.
You don't need to deploy the supported LLMs separately for model router, except Claude models, which you deploy from the model catalog first.
The other options, and when you'd use them
- Deploy each model separately: Not needed for the supported models. Only Claude models need a separate deployment.
- Cost mode: It picks the most cost-effective model within a wider quality band. That's the opposite of "regardless of cost".
- Keep Balanced: Balanced also optimizes for cost within a small quality band, so it doesn't always pick the top model.
Exam tip
Router: max quality = Quality mode. Long prompts = model subset (the smallest model sets the context window). Separate deployment only for Claude.
References
Last reviewed: