Implement generative AI and agentic solutions 2.2 Build agents by using Foundry
You run an offline model evaluation of a RAG chat application on a dataset of queries and…
Study guide skill: Integrate monitoring into deployed agents, evaluate agent behavior, and perform error analysis
Question B2-017
You run an offline model evaluation of a RAG chat application on a dataset of queries and responses. You want to measure whether its responses fell for jailbreak instructions planted inside retrieved documents. Which built-in evaluator should you add?
Select one answer.
Answer: C
The Indirect Attack (XPIA) evaluator, builtin.indirect_attack, measures to what extent a response fell for an indirect jailbreak attempt, meaning attacks injected into the context of a document or source. It's listed for model evaluations only, not agent targets, which matches this dataset evaluation. Groundedness checks fabrication against context, Task Adherence checks rule compliance, and Protected Material looks for copyrighted text.
Deep dive
How it works
Foundry's risk and safety evaluators score responses for specific harms. The one built for this scenario is Indirect Attack (XPIA), builtin.indirect_attack:
- It measures to what extent the response fell for an indirect jailbreak attempt.
- Indirect attacks are jailbreaks injected into the context of a document or source, which can cause altered or unexpected model behavior. They're also called cross-domain prompt injected attacks (XPIA).
- In a RAG app, that's exactly a planted instruction inside a retrieved document.
Two practical points from the evaluator list:
- The target column says Model only. It's for model and dataset evaluations, which matches an offline run on queries and responses.
- Its inputs are query and response, or messages, so your dataset only needs the conversation the app produced.
Like the other safety evaluators, results roll up into an aggregate defect rate: the percentage of responses where undesired content was detected.
The other options, and when you'd use them
- Task Adherence: It asks whether the agent followed the rules and constraints in its instructions. It's an agent quality check, not injection detection.
- Protected Material: It measures text under copyright, such as song lyrics, recipes and articles.
- Groundedness: It measures how well the response aligns with the given context without fabricating content. A response can be well grounded in a poisoned document.
Exam tip
"Did the response follow instructions hidden in documents?" = Indirect Attack (XPIA), builtin.indirect_attack. Direct user jailbreaks are a different evaluator.
References
Last reviewed: