Implement generative AI and agentic solutions 2.3 Optimize and operationalize generative AI systems
When you use the Chat Completions API, how should you provide few-shot examples that show…
Study guide skill: Tune generation behavior, such as prompt engineering and adjusting model parameters
Question B1-021
When you use the Chat Completions API, how should you provide few-shot examples that show the model the expected output format?
Select one answer.
Answer: C
In the Chat Completions API, instructions go in the system message and few-shot examples are given as example user and assistant exchanges that prime the model. Few-shot learning conditions the model through the prompt; it doesn't permanently change it like fine-tuning.
Deep dive
How it works
Few-shot learning means including one or more examples of the behavior you want, usually as input and output pairs, so the model can see the pattern. It isn't learning in the sense that the model is permanently changed. The examples only condition the model for the current inference. A prompt with no examples is called "zero-shot."
In the Chat Completions API, the structure is specific:
- The system message carries the instructions.
- Few-shot examples come next, as a series of example user and assistant messages placed after the initial system message.
- The real user message comes last.
The docs say these example exchanges can prime the model to respond in a certain way, emulate particular behaviors and seed answers to common questions. The headline-to-topic example on the page shows the payoff: without examples the model seems to guess, while with a few examples it labels new headlines cleanly, even inferring a label it never saw in the examples. Example exchanges in the expected format are the most direct way to teach the model an output shape.
The other options, and when you'd use them
- The stop parameter: The API reference describes stop as up to 4 sequences where the API will stop generating further tokens. It controls where output ends, so it can't show the model what the output should look like.
- logit_bias values: logit_bias modifies the likelihood of specified tokens appearing in the completion, mapping token IDs to bias values. It adjusts individual tokens, so it isn't a way to give examples of a whole output format.
- A fine-tuning job before each request: Few-shot examples aren't learning in the sense that the model is permanently changed; they condition the model for the current inference. Fine-tuning is a separate training job that runs from your data and settings, so it isn't how you supply examples within a Chat Completions request.
Exam tip
"Show the model examples in Chat Completions" means example user/assistant message pairs after the system message and before the real question. "Permanently change behavior" points to fine-tuning instead.
References
Last reviewed: