Implement generative AI and agentic solutions 2.1 Build generative applications by using Foundry
A downstream system will reject any response that doesn't exactly match a JSON Schema…
Study guide skill: Design workflows, tool-augmented flows, and multistep reasoning pipelines
Question B1-018
A downstream system will reject any response that doesn't exactly match a JSON Schema with required fields and no extra properties. Which approach should you use with an Azure OpenAI model?
Select one answer.
Answer: A
Structured outputs make the model follow a JSON Schema that you provide in the request, while the older JSON mode only guarantees valid JSON, not schema adherence. Instructions in the prompt or a lower temperature make compliance more likely but don't enforce the schema.
Deep dive
How it works
Structured outputs make a model follow a JSON Schema that you provide in the inference API call. Both APIs support it: with Chat Completions you put the schema in response_format, and with Responses you put it in text.format. The docs recommend it for function calling, extracting structured data and building complex multi-step workflows. For function calling, you turn it on with a single parameter, strict: true.
The schema has rules you need to know. All fields must be required. To emulate an optional field, use a union type with null, such as "type": ["string", "null"]. The documented examples also set "additionalProperties": false, and a root object can't be of type anyOf. There are two caveats. Structured outputs aren't supported with parallel function calls, so set parallel_tool_calls to false. And they currently aren't supported in the bring-your-own-data scenario.
The other options, and when you'd use them
- JSON mode: JSON mode guarantees valid JSON, but it doesn't guarantee the output matches a particular schema. It's still supported, though the docs recommend structured outputs where possible. If you do use it, include the word "JSON" in the messages, typically in the system message. Otherwise the request fails with a 400 error.
- A lower temperature: Lower values, such as 0.2, make output more focused and concrete, and the docs suggest a much lower temperature for something like a legal document. Change either temperature or top_p, not both. Neither one enforces a schema.
- A system message that says "reply only with JSON": Instructions guide the model but enforce nothing. The schema guarantee comes from the structured outputs setting in the API call. This is the same reason the docs contrast structured outputs with JSON mode, which "couldn't ensure strict adherence to the supplied schema".
Exam tip
"Must match the schema exactly" means structured outputs with strict (response_format or text.format). "Just valid JSON" means JSON mode. Remember that all fields must be required, optional fields use a union with null, and parallel_tool_calls must be false.
References
Last reviewed: