Plan and manage an Azure AI solution 1.1 Choose the appropriate Foundry services for generative AI and agents

A support assistant must (1) answer questions about return policies that change every…

Study guide skill: Choose an appropriate model for each task, including large language models (LLMs), small language models, multimodal models, and Foundry Tools

Question B8-001

A support assistant must (1) answer questions about return policies that change every week and (2) always reply in the company's tone and a fixed response structure. The team has many high-quality example conversations. Which two choices fit? (Choose two.)

Select 2 answers.

  • A Ground the assistant on the current policy documents through retrieval, because fine-tuning doesn't replace retrieval for current information ✓ Correct answer
  • B Rely on the fine-tuned model instead of application-level safety controls
  • C Use preference fine-tuning (DPO), because it needs only one correct response per input
  • D Use supervised fine-tuning (SFT) with the example conversations to align tone and structure ✓ Correct answer
  • E Fine-tune the model again every week so that the new policies are stored in its weights

Answer: A, D

Fine-tuning can align style and structure, and SFT fits when you can demonstrate the desired behavior with prompts and responses. Fine-tuning doesn't replace retrieval for current information or application-level safety controls, so weekly policy changes belong in retrieval. DPO trains on preferred and rejected responses to the same input, for qualities like tone rather than one correct answer.

Deep dive

How it works

This scenario has two needs that call for two different tools.

Weekly-changing policy facts: retrieval. Microsoft is explicit that fine-tuning doesn't replace retrieval for current information, and it doesn't replace application-level safety controls either. Fine-tuning also adds training and hosting costs. Policies that change every week belong in a grounding source the assistant retrieves from at query time.

Consistent tone and structure: fine-tuning. One of the listed reasons to fine-tune is to align style and structure, so responses follow a consistent tone, format or schema. The team has many high-quality example conversations, which fits supervised fine-tuning (SFT). Its training data is prompts and desired responses, and you use it when you can demonstrate the behavior the model should learn.

Even after fine-tuning, Microsoft says to keep application-level safety controls and to evaluate the final serving runtime.

The other options, and when you'd use them

  • Rely on the fine-tuned model instead of safety controls: Fine-tuning doesn't replace application-level safety controls, and the guidance says to keep them.
  • DPO because it needs only one correct response per input: Backwards. DPO trains on preferred and rejected responses to the same input. It fits when quality depends on preferences such as tone or completeness rather than one correct answer.
  • Re-fine-tune weekly to store policies in the weights: That's using fine-tuning for current information, which it doesn't replace retrieval for, and it adds training and hosting costs each time.

Exam tip

Facts that change = retrieval. Tone, format, structure = SFT with examples. DPO = preferred vs rejected pairs. Safety controls stay either way.

References

Last reviewed:

Ready to test yourself? The quiz asks this and every other AI-103 question, with instant feedback. Practice this in a quiz →

All Plan and manage questions · AI-103 study guide

Thanks for your feedback!