haystack-pipeline

Solid

Haystack NLP pipeline configuration for document processing and QA

Data & Documents 1,160 stars 71 forks Updated today MIT

Install

View on GitHub

Quality Score: 94/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
51
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Haystack Pipeline Skill ## Capabilities - Configure Haystack pipeline components - Set up document stores and retrievers - Implement reader/generator models - Design custom pipeline graphs - Configure preprocessing pipelines - Implement evaluation pipelines ## Target Processes - rag-pipeline-implementation - intent-classification-system ## Implementation Details ### Core Components 1. **DocumentStores**: Elasticsearch, Weaviate, FAISS, etc. 2. **Retrievers**: BM25, Dense, Hybrid 3. **Readers/Generators**: Extractive and generative QA 4. **Preprocessors**: Document cleaning and splitting ### Pipeline Types - Retrieval pipelines - RAG pipelines - Evaluation pipelines - Indexing pipelines ### Configuration Options - Component selection - Pipeline graph design - Document store backend - Model selection - Preprocessing settings ### Best Practices - Modular pipeline design - Proper preprocessing - Evaluation integration - Component versioning ### Dependencies - haystack-ai - farm-haystack (legacy)

Details

Author
a5c-ai
Repository
a5c-ai/babysitter
Created
4 months ago
Last Updated
today
Language
JavaScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

ai-data-engineering

Data pipelines, feature stores, and embedding generation for AI/ML systems. Use when building RAG pipelines, ML feature serving, or data transformations. Covers feature stores (Feast, Tecton), embedding pipelines, chunking strategies, orchestration (Dagster, Prefect, Airflow), dbt transformations, data versioning (LakeFS), and experiment tracking (MLflow, W&B).

368 Updated 5 months ago
ancoleman
Data & Documents Featured

data-engineering-data-pipeline

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

39,350 Updated today
sickn33
Data & Documents Listed

data-engineering-data-pipeline

You are a data pipeline architecture expert specializing in scalable, reliable, and cost-effective data pipelines for batch and streaming data processing.

335 Updated today
aiskillstore
Data & Documents Solid

doc-pipeline

Chain document operations into reusable pipelines

364 Updated today
majiayu000
Data & Documents Listed

pipeline-architect

Designs and implements data pipelines: ETL/ELT, streaming, batch processing, schema migrations, and data warehouse architecture. Covers Kafka, Airflow, dbt, Spark, ClickHouse, BigQuery, Snowflake, Redis Streams, and more. Use this skill when the user asks about data pipelines, ETL jobs, data transformation, streaming setup, data warehouse design, CDC, schema migrations, data quality checks, or anything involving moving data from source to target. Also triggers on "build a pipeline," "migrate data from X to Y," "set up streaming," "design my data warehouse," or "data quality is bad, help me fix it."

1 Updated 4 days ago
mturac