Implement computer vision solutions 3.2 Design and implement multimodal understanding workflows
A sports broadcaster uses Content Understanding video analysis to find the moment a ball…
Study guide skill: Implement solutions that identify objects, components, or regions within images or video
Question B5-007
A sports broadcaster uses Content Understanding video analysis to find the moment a ball crosses the goal line and to read the jersey numbers of distant players. It handles scene-level descriptions well but often misses single-frame events and small numbers. What is the most likely cause?
Select one answer.
Answer: D
Content Understanding video analysis inspects about one frame per second and resizes sampled frames to 512 x 512 pixels, so the docs warn that rapid motions or single-frame events might be missed and small text or distant objects can be lost. With segmentation off, the whole video is still analyzed as a single segment. Only spoken words are transcribed (music, sound effects, and ambient noise are ignored), and every shot gets at least one sampled key frame rather than only its first frame.
Deep dive
How it works
Content Understanding video analysis is built for scene-level understanding, not frame-accurate inspection. The video overview lists the limits to keep in mind:
- Frame sampling, about 1 FPS: the analyzer inspects about one frame per second. Rapid motions or single-frame events might be missed.
- Frame resolution, 512 x 512 px: sampled frames are resized to 512 pixels square. Small text or distant objects can be lost.
- Speech: only spoken words are transcribed. Music, sound effects and ambient noise are ignored.
A ball crossing the goal line is a single-frame event that falls between samples. A jersey number on a distant player is small detail that a 512 x 512 downscale wipes out. Scene descriptions come out fine because they don't depend on either.
For goal-line decisions or reading numbers, pair Content Understanding (summaries, chapters, metadata) with a purpose-built pipeline. For example, extract full-resolution frames around the moment and analyze those.
The other options, and when you'd use them
- Segmentation is disabled: With enableSegment false, the service treats the whole video as a single segment and still extracts metadata across its full duration. Frame analysis isn't turned off.
- Crowd noise confuses the visual analysis: Only spoken words are transcribed. Ambient noise is ignored.
- Only the first frame of each shot: Keyframes are selected uniformly from each camera shot, and each shot gets at least one, not just its first frame.
Exam tip
Content Understanding video: about 1 FPS sampling and 512 x 512 frames. Expect missed single-frame events and lost small text or distant objects.
References
Last reviewed: