Implement generative AI and agentic solutions 2.3 Optimize and operationalize generative AI systems
Your prompt contains long instructions followed by a large amount of grounding text.
Study guide skill: Tune generation behavior, such as prompt engineering and adjusting model parameters
Question B1-020
Your prompt contains long instructions followed by a large amount of grounding text. The model often ignores the instructions near the top. Which prompt-engineering technique is recommended to try?
Select one answer.
Answer: A
Models can show recency bias, so information at the end of a prompt may influence the output more than information at the start. The guidance suggests experimenting with repeating instructions at the end and evaluating the effect. Raising temperature or splitting deployments doesn't address this.
Deep dive
How it works
Microsoft's prompt-engineering guidance names this problem directly: models can be susceptible to recency bias. Here that means information at the end of the prompt might influence the output more than information at the beginning. When long instructions are followed by a large block of grounding text, the instructions end up far from the end, and the model may underweight them.
The recommended technique is to repeat the instructions at the end of the prompt and then evaluate the effect on the response. The docs frame it as something worth experimenting with, not a guaranteed fix, so measure before and after.
Two related techniques from the same page work well alongside it:
- Start with clear instructions: The docs still generally recommend telling the model the task at the beginning of the prompt, before the context or examples. They also note that in their testing, newer chat models responded the same whether or not this technique was used. Repeating the instructions at the end complements this.
- Prime the output: End the prompt with a few words or a cue, such as "Here's a bulleted list of key points:", so the response starts in the form you want.
In the Chat Completions API, instructions usually live in the system message, so in practice you'd keep them there and also restate the key ones after the grounding text.
The other options, and when you'd use them
- Move all instructions into the user message and remove the system message: This doesn't change where the instructions sit relative to the long grounding text. The docs place instructions in the system message for the Chat Completions API.
- Split the grounding text across multiple deployments: Deployments host models. Spreading context across them breaks the single prompt the model needs and doesn't address recency bias.
- Increase the temperature: The temperature parameter controls randomness. A higher value makes output more random and divergent, which makes instruction-following less consistent, not more.
Exam tip
"Model ignores instructions at the top of a long prompt" means recency bias, so repeat the instructions at the end. Temperature changes randomness, not attention to instructions.
References
Last reviewed: