Implement generative AI and agentic solutions 2.2 Build agents by using Foundry
For each test task you have the expected sequence of tool calls that an expert would make.
Study guide skill: Integrate monitoring into deployed agents, evaluate agent behavior, and perform error analysis
Question B4-025
For each test task you have the expected sequence of tool calls that an expert would make. A support agent usually reaches the right outcome but takes extra steps compared with those expected sequences. Which agent evaluator measures this against your ground truth?
Select one answer.
Answer: A
Task Navigation Efficiency compares the agent's tool calls with expected tool sequences (ground truth) to judge whether it took an efficient path, which fits reducing unnecessary steps. Task Completion checks whether the agent delivered a usable result that meets the user's requirements, which this agent already does. Task Adherence checks compliance with rules and policy constraints, and Tool Call Success checks for technical failures in tool execution.
Deep dive
How it works
Foundry's agent evaluators split "did it work?" from "did it work well?":
- Task Completion (preview): did the agent complete the requested task with a usable deliverable that meets all user requirements?
- Task Adherence (preview): did the agent's actions follow its assigned tasks according to rules, procedures and policy constraints?
- Task Navigation Efficiency: did the agent make its tool calls efficiently, compared with expected tool sequences?
Task Navigation Efficiency is the one built for this problem. Microsoft lists its use as optimizing agent workflows, reducing unnecessary steps and validating against known optimal paths, and it requires ground truth: you pass the expected actions (expected_actions, with an optional matching_mode) alongside the agent's actions or messages. It returns Pass/Fail.
Your agent passes on outcome but takes extra steps, and you have expert tool sequences, so this is the evaluator to use.
The other options, and when you'd use them
- Tool Call Success: It detects technical failures such as API errors, timeouts or exceptions. Extra steps that succeed aren't failures.
- Task Completion (preview): It checks for a usable result that meets requirements, which the agent already delivers.
- Task Adherence (preview): It checks compliance with rules, procedures and policy constraints, not path efficiency.
Exam tip
Right result but too many steps, with expected tool sequences = Task Navigation Efficiency.
References
Last reviewed: