A RAG chatbot sometimes adds facts that don't appear in the retrieved documents.

How it works

Foundry's RAG evaluators split into system evaluation, which judges the final response, and process evaluation, which judges the retrieval step. Groundedness is a system evaluator. It measures how well the generated response aligns with the given context without fabricating content, which the docs call the precision aspect. A chatbot adding facts that aren't in the retrieved documents is exactly what it catches.

Groundedness uses an LLM judge, so you pass a deployment_name. Its inputs are the response plus context (optional, but recommended), and adding the query can improve scoring. It returns Pass/Fail against a threshold on a 1-5 scale. For agent evaluation, context is optional when the response contains tool call messages, because the evaluator can pull context from the tool results, and providing tool_definitions helps it judge tool-assisted answers. If you want a stricter definition, Groundedness Pro (preview) uses Azure AI Content Safety and returns True/False.

The docs give a handy way to think about it. Groundedness is precision: the answer contains nothing outside the grounding context. Response completeness is recall: the answer doesn't miss critical information.

The other options, and when you'd use them

• Response Completeness (preview): This measures how completely the response covers the expected information compared with ground truth (recall), so it needs ground_truth. Use it when the problem is answers that leave out critical facts.
• Fluency: This measures grammatical accuracy, vocabulary range, sentence complexity and readability, independent of factual correctness. Use it when writing quality matters regardless of content.
• Document Retrieval: This is a process evaluator. It compares retrieved documents with ground-truth relevance labels, using metrics such as NDCG, XDCG, Fidelity, Max Relevance and Holes. Use it when retrieval quality is the bottleneck and you have query relevance labels.

Exam tip

"Adds facts that aren't in the sources" means Groundedness. "Leaves out key facts" means Response Completeness, "search returns the wrong documents" means Retrieval or Document Retrieval, and "reads badly" means Fluency or Coherence.

References

• Retrieval-Augmented Generation (RAG) evaluators
• General purpose evaluators