Retrieval Quality vs. Faithfulness
Also: retrieval metrics vs. generation metrics · RAG eval split · grounding vs. retrieval
Separate whether RAG fetched the right context (retrieval) from whether the answer stayed true to it (faithfulness).
Why CCAR-P tests this
The exam checks that you diagnose RAG failures by evaluating retrieval and generation as distinct stages.
A RAG pipeline fails in two independent places, and evaluating them separately is what lets you fix the right one. Retrieval quality asks: did we fetch the correct context? It is measured with information-retrieval metrics like recall (did we get all the relevant chunks?) and precision (were the fetched chunks actually relevant?), plus rank-aware variants. Poor retrieval means the model never had the facts.
Faithfulness (a.k.a. groundedness) asks: is the generated answer supported by the retrieved context, with no fabricated or contradicted claims? It is a property of the generation step, not retrieval. Separately, answer relevance asks whether the response actually addresses the user's question. A model can be perfectly faithful to retrieved text yet still unhelpful, or fluent and relevant yet unfaithful (hallucinating beyond the sources).
Because the failures are independent, high retrieval recall with low faithfulness points at the prompt or model, while low recall points at chunking, embeddings, or the retriever. Collapsing them into one "RAG score" hides which stage broke.
Exam trap
Retrieval recall/precision and generation faithfulness are different metrics on different stages. A high retrieval score does not imply the answer is grounded.
Commonly confused with
Independent, unofficial study material from Claude Cert Prep. Not affiliated with Anthropic.