Plan and manage an Azure AI solution 1.3 Manage, monitor, and secure AI systems
An operations team watches a production agent on the Monitor tab (Agent Monitoring…
Study guide skill: Monitor model performance, drift, safety events, and grounding quality
Question B6-009
An operations team watches a production agent on the Monitor tab (Agent Monitoring Dashboard) in Foundry, and recurring evaluations already run on a schedule. The team now wants to be notified automatically when latency or evaluation scores degrade, and wants scheduled adversarial tests that look for risks such as data leakage. Which two Monitor settings should they configure? (Choose two.)
Select 2 answers.
Answer: A, E
In Monitor settings, Alerts (preview) detect performance anomalies, evaluation failures, and security risks and can be set for latency, token usage, evaluation scores, or red team findings. Red team scans (preview) run adversarial tests on a schedule to detect risks such as data leakage or prohibited actions. Recurring evaluations validate performance against benchmarks, and the token usage chart and run success rate card only display metrics.
Deep dive
How it works
The Monitor tab in Foundry (the Agent Monitoring Dashboard) has a settings panel with several monitoring features. Each one does a different job:
- Recurring evaluations (preview): run evaluations on a schedule to validate performance against benchmarks. The team already has these.
- Red team scans (preview): run adversarial tests to detect risks such as data leakage or prohibited actions. You enable them, pick an evaluation template and set a schedule.
- Alerts (preview): detect performance anomalies, evaluation failures and security risks. You configure alerts for latency, token usage, evaluation scores or red team findings.
The team's two new needs map one-to-one: automatic notification on latency or score drops is Alerts, and scheduled adversarial testing is Red team scans. Alerts can also fire on red team findings, so the two work together.
The dashboard's metrics are for reading, not acting. Token usage shows token counts for agent traffic in the selected time range, and run success rate is the percentage of runs that complete successfully.
The other options, and when you'd use them
- A second recurring evaluation: Recurring evaluations validate performance against benchmarks on a schedule. They don't send notifications and aren't adversarial tests.
- Token usage chart: It shows token counts for the selected time range. It's a metric display, not a notification.
- Run success rate card: Also a metric: the percentage of runs that complete successfully.
Exam tip
Monitor settings: notify = Alerts. Adversarial on a schedule = Red team scans. Benchmarks on a schedule = Recurring evaluations. Charts and cards only display.
References
Last reviewed: