RAG pipelines that survive production

Retrieval quality is a data problem

Most underperforming RAG systems are not failing at generation — they are failing at retrieval. The documents are scanned PDFs with no structure, chunks are cut at arbitrary token counts, and nothing records which customer, product or time period a chunk belongs to.

Fix the pipeline before the model: clean extraction, structure-aware chunking (respect headings and tables), and rich metadata attached to every chunk. Then filter at query time by that metadata instead of hoping the vector search sorts it out.

Evaluate retrieval separately

Measure whether the right chunks are retrieved (recall@k) before you ever measure answer quality. A generation model cannot answer from context it never received. Keep a golden set of question-to-source mappings and run it on every pipeline change — embedding model swaps especially.

Ship with citations and feedback loops

Every production answer should carry citations users can check, and a thumbs-down path that captures the failing query verbatim. That feedback stream becomes your next round of evaluation cases. RAG systems are never 'done' — they are maintained like search engines, not shipped like features.