Implement computer vision solutions 3.3 Implement responsible AI for multimodal content
A social platform must screen memes in which the harmful message is written as text…
Study guide skill: Enforce visual policy rules, such as applying watermarks, flagging prohibited symbols, upholding brand usage requirements, and detecting potentially inappropriate content
Question B5-009
A social platform must screen memes in which the harmful message is written as text inside the image, often next to a caption that looks harmless on its own. Moderation must judge the image, the text in it, and the caption together. Which Azure AI Content Safety approach fits? (The multimodal API is currently in preview.)
Select one answer.
Answer: D
The multimodal API (preview) analyzes an image together with its associated text, and with enableOcr set to true it also runs OCR and analyzes the text it detects in the image at the same time. Analyzing the image and its text together preserves context that separate image and text calls lose. Turning OCR off would skip the text inside the image, and Prompt Shields detects adversarial inputs to LLMs rather than harmful content in images.
Deep dive
How it works
The Content Safety Multimodal API (preview) analyzes content that contains both an image and text. The docs explain why that matters: analyzing an image together with its associated text can preserve context and give a more complete understanding of the content. That's the meme problem, where a harmless-looking caption and the words inside the image only become harmful together.
You call the imageWithText:analyze operation with:
- image: base64 content or a blob URL (not both). Images can be up to 7,200 x 7,200 pixels and 4 MB.
- text: the caption attached to the image, up to 1,000 characters.
- enableOcr: when set to true, the service runs OCR and analyzes the detected text along with the image at the same time. It recognizes up to 1,000 characters from the image and truncates the rest.
- categories: optional. If you don't specify any, all four harm categories are used.
The response lists a severity for each category. The multimodal model supports the full 0-7 severity scale. In the portal's try-it page, the service analyzes the image content, any text that appears in the image, and the text you associate with it.
The other options, and when you'd use them
- Send the caption to Prompt Shields: Prompt Shields scans text for user input attacks on an LLM. Harm categories such as hate and violence are rated by the analyze APIs.
- enableOcr set to false: OCR is the part that reads the text inside the image. Turning it off removes the very signal you need.
- Separate image and text calls, keep the higher severity: The separate Analyze image and Analyze text APIs each scan for the four harm categories. Scoring them apart loses the context that comes from analyzing the image and its text together.
Exam tip
"Text inside the image plus a caption, judged together" means the multimodal API with enableOcr set to true.
References
Microsoft Learn references
Last reviewed: