build-rag-pipelinelisted

# Build a RAG Pipeline ## When this applies - "Let the model answer questions about our documentation" - Semantic or hybrid search over a corpus - An existing vector search needs to become a real pipeline ## Do not use for - Diagnosing a specific wrong answer → `diagnose-rag-failure` - Building the labelled eval set → `build-rag-evalset` - Multi-hop questions requiring an agent → see `../blueprints/blueprints/agentic-rag.md` ## Inputs to collect first | Input | Why needed | Default if unspecified | |---|---|---| | Corpus size and format | Decides store and parser | Ask — this changes everything | | Multi-tenant? | Tenancy is a correctness requirement, not a feature | Assume yes; build the filter in | | Update frequency | Batch vs incremental indexing | Daily batch | | Query examples (5–10 real ones) | Reveals whether exact-match matters | Ask — do not skip this | | Latency budget | Reranking and transformation fit or don't | 2 s end to end | **Check first:** if the whole corpus is under ~100k tokens and rarely changes, put it in the prompt with caching instead. Say so and stop — it will beat a new RAG system on quality and effort. ## Procedure ### Step 1 — Parse, preserving structure Extract text **plus** heading hierarchy, page numbers, and table boundaries. A parser that flattens structure caps the quality of everything downstream. **Stop condition:** parsed output for 3 sample documents retains headings and readable tables. ### Step 2 — Chunk structurally, with