RAG Pipeline

A RAG pipeline has two phases. Indexing (offline): documents are loaded, split via a chunking strategy, embedded into vectors, and stored in a vector/hybrid index. Retrieval + generation (online): the user query is embedded, the top-k relevant chunks are fetched (often then reranked), and those chunks are injected into the prompt as grounding context for Claude to answer over.

RAG reduces hallucination and keeps knowledge fresh without retraining, but it introduces its own failure surface: bad chunking, weak retrieval, or stale indexes produce confidently wrong answers. Retrieval quality and generation quality are separate concerns and must be measured separately.

Evaluate retrieval with metrics like recall@k, precision@k, and MRR/nDCG — did the right chunks come back? Evaluate generation with faithfulness/groundedness — did the answer stay true to the retrieved context, or did it drift? A pipeline can retrieve perfectly and still hallucinate, or generate faithfully over the wrong documents.