dspy-rag-pipeline

# DSPy RAG Pipeline ## Goal Build retrieval-augmented generation pipelines with ColBERTv2 that can be systematically optimized. ## When to Use - Questions require external knowledge - You have a document corpus to search - Need grounded, factual responses - Want to optimize retrieval + generation jointly ## Related Skills - Optimize this pipeline: [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md), [dspy-bootstrap-fewshot](../dspy-bootstrap-fewshot/SKILL.md) - Build local semantic retrieval: [dspy-embedding-retrieval](../dspy-embedding-retrieval/SKILL.md) - Evaluate results: [dspy-evaluation-suite](../dspy-evaluation-suite/SKILL.md) - Design signatures: [dspy-signature-designer](../dspy-signature-designer/SKILL.md) ## Inputs | Input | Type | Description | |-------|------|-------------| | `question` | `str` | User query | | `k` | `int` | Number of passages to retrieve | | `rm` | `dspy.Retrieve` | Retrieval model (ColBERTv2) | ## Outputs | Output | Type | Description | |--------|------|-------------| | `context` | `list[str]` | Retrieved passages | | `answer` | `str` | Generated response | ## Workflow ### Phase 1: Configure Retrieval ```python import dspy # Configure LM and retriever colbert = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts') dspy.configure( lm=dspy.LM("openai/gpt-4o-mini"), rm=colbert ) ``` ### Phase 2: Define Signature ```python class GenerateAnswer(dspy.Signature): """Answer questions with short factoid ans...