# Retrieval Quality vs. Faithfulness

Separate whether RAG fetched the right context (retrieval) from whether the answer stayed true to it (faithfulness).

## Why the CCAR-P exam tests this

The exam checks that you diagnose RAG failures by evaluating retrieval and generation as distinct stages.

## Definition

A RAG pipeline fails in two independent places, and evaluating them separately is what lets you fix the right one. **Retrieval quality** asks: did we fetch the correct context? It is measured with information-retrieval metrics like **recall** (did we get all the relevant chunks?) and **precision** (were the fetched chunks actually relevant?), plus rank-aware variants. Poor retrieval means the model never had the facts.

**Faithfulness** (a.k.a. groundedness) asks: is the generated answer supported by the retrieved context, with no fabricated or contradicted claims? It is a property of the **generation** step, not retrieval. Separately, **answer relevance** asks whether the response actually addresses the user's question. A model can be perfectly faithful to retrieved text yet still unhelpful, or fluent and relevant yet **unfaithful** (hallucinating beyond the sources).

Because the failures are independent, high retrieval **recall** with low **faithfulness** points at the prompt or model, while low recall points at chunking, embeddings, or the retriever. Collapsing them into one "RAG score" hides which stage broke.

## Exam trap

Retrieval recall/precision and generation faithfulness are different metrics on different stages. A high retrieval score does not imply the answer is grounded.

## Commonly confused with

- **llm-as-judge** — LLM-as-judge is often the tool that scores faithfulness; faithfulness is the property, not the scorer.
- **golden-dataset** — RAG eval needs reference contexts and answers in the golden set; this concept is about which metric measures which stage.

## Related

- llm-as-judge
- golden-dataset
- eval-driven-development
- regression-testing

---

Source: Claude Cert Prep — an independent, unofficial CCAR-P study resource (domain P4). Not affiliated with Anthropic.