Implement generative AI and agentic solutions 2.1 Build generative applications by using Foundry
A RAG app's answers rarely contain unsupported facts, but subject-matter experts say the…
Study guide skill: Evaluate models and apps, including detecting fabrications, relevance, quality, and safety
Question B3-016
A RAG app's answers rarely contain unsupported facts, but subject-matter experts say the answers often leave out critical information that appears in their expected answers. You have a test dataset with a ground-truth answer for each query. Which built-in evaluator directly measures this problem?
Select one answer.
Answer: D
Response Completeness (preview) measures how completely a response covers the expected information in the ground truth, which is the recall side of answer quality. Groundedness measures precision: whether the response stays within the grounding context, which isn't the problem here. Relevance judges how well a response addresses the query without ground truth, and Retrieval judges the relevance of retrieved chunks rather than the final answer.
Deep dive
How it works
RAG quality has two sides, and Foundry has an evaluator for each:
- Groundedness is the precision side: the response doesn't contain content outside the grounding context.
- Response Completeness (preview) is the recall side: the response doesn't miss critical information compared to the expected response or ground truth.
The symptom here is pure recall: answers rarely invent facts (precision is fine), but they leave out information that the experts' expected answers contain. Response Completeness measures how completely the response covers the expected information compared to ground truth, and it needs exactly what you have: a ground-truth answer per query. It returns Pass/Fail based on a threshold on a 1-5 scale.
The other options, and when you'd use them
- Retrieval: It judges how relevant the retrieved context chunks are to the query. That's the search step, not whether the final answer covers the ground truth.
- Groundedness: It measures whether the response stays within the context without fabricating content (precision), which is already fine here.
- Relevance: It assesses how well the response addresses the query when you don't have ground truths. You do have them, and the problem is missing content, not off-topic answers.
Exam tip
Makes things up = Groundedness (precision). Leaves things out vs ground truth = Response Completeness (recall).
References
Last reviewed: