Reranking

Reranking is a two-stage retrieval pattern. Stage one uses a fast, cheap retriever (vector and/or keyword) to fetch a wide candidate set — say top-50. Stage two applies a slower, more accurate reranker (typically a cross-encoder that scores the query and each candidate jointly) to reorder them, and you keep the top few (say top-5) for the prompt.

The payoff is precision: bi-encoder vector search embeds query and document separately and can miss fine-grained relevance, while a cross-encoder reads them together and judges actual match quality. This directly lifts the answer because LLMs weight earlier/most-relevant context more heavily and the prompt has limited room.

The cost is latency and compute per candidate, so you rerank a bounded candidate set, not the whole corpus. Reranking improves ordering; it cannot recover a relevant document that stage-one retrieval never returned — fix recall first, then precision.