grepai-chunkinglisted

# GrepAI Chunking Configuration This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase. ## When to Use This Skill - Optimizing search accuracy - Adjusting for code style (verbose vs. concise) - Troubleshooting search results - Understanding how indexing works ## What is Chunking? Chunking is the process of splitting source files into smaller segments for embedding: ``` ┌─────────────────────────────────────┐ │ Large Source File │ │ (1000+ tokens) │ └─────────────────────────────────────┘ ↓ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │ │ ~512 │ │ ~512 │ │ ~512 │ │ tokens │ │ tokens │ │ tokens │ └─────────┘ └─────────┘ └─────────┘ ↓ Each chunk gets its own embedding ``` ## Why Chunking Matters Embedding models have optimal input sizes: - **Too large chunks:** Less precise search results - **Too small chunks:** Lost context, fragmented results - **Just right:** Good balance of precision and context ## Configuration ### Basic Settings ```yaml # .grepai/config.yaml chunking: size: 512 # Tokens per chunk overlap: 50 # Overlap between chunks ``` ### Understanding Parameters #### Chunk Size The target number of tokens per chunk. | Size | Effect | |------|--------| | 256 | More precise, less context | | 512 | Balanced (default) | | 1024 | More context, less preci