Implement generative AI and agentic solutions 2.3 Optimize and operationalize generative AI systems
A reasoning model handles a high volume of simple classification requests and responses…
Study guide skill: Implement model reflection, chain-of-thought evaluations, and self-critique loops
Question B1-023
A reasoning model handles a high volume of simple classification requests and responses are slower and more expensive than needed. Which change most directly reduces how much the model thinks before answering?
Select one answer.
Answer: C
The reasoning_effort parameter controls how much the model thinks before it answers, and reasoning tokens are billed as output, so lower effort cuts latency and cost for simple tasks. A developer message works the same as a system message, and raising the token cap allows more output rather than less.
Deep dive
How it works
Reasoning models generate reasoning tokens on top of the usual input and output tokens. They use them to work through the prompt: breaking the problem apart, weighing approaches and dropping paths that don't hold up. Reasoning tokens never appear in the message content, but they take up space in the context window and are billed as output tokens. So a request that thinks longer costs more, even when the visible answer is short.
The reasoning_effort parameter controls how much the model thinks before it answers. The higher the effort, the longer the model spends on the request, which generally produces more reasoning tokens. Supported values and defaults vary by model, and the docs map them to workloads:
- none: latency-critical work that doesn't benefit from reasoning, such as voice, fast information retrieval and classification.
- low: efficient reasoning with a modest latency increase, for tool use, planning and search where speed and cost matter.
- medium: a balanced starting point for most workloads.
- high: hard reasoning and deep planning, where quality matters more than latency.
For a high volume of simple classification calls, lowering reasoning_effort is the direct lever on both latency and cost. You can check completion_tokens_details.reasoning_tokens in the response to confirm that fewer reasoning tokens are being spent.
The other options, and when you'd use them
- Increase max_completion_tokens: This caps the total the model generates, including reasoning, visible output and formatting tokens. Raising it allows more output, not less thinking.
- Add a developer message: Developer messages are functionally the same as system messages. They give instructions; they don't control reasoning depth.
- Raise the temperature: Temperature affects randomness in sampling, not how much the model reasons. It doesn't cut reasoning tokens.
Exam tip
"Reasoning model is slow and expensive on easy tasks" means lower reasoning_effort. Use max_completion_tokens (Chat Completions) or max_output_tokens (Responses) to cap total spend.
References
Last reviewed: