ml-pipeline

# ML Pipeline ## Purpose Build a machine learning pipeline that produces the same model twice and behaves in production the way it did in training. The two defining failure modes are irreproducible training and train/serve skew — the model sees different features in production than it saw in training, and quietly degrades. ## When to Use - Building a training or inference pipeline. - A model that performed well offline and poorly in production. - Setting up monitoring for a deployed model. - Establishing a retraining cadence. ## Capabilities - Feature engineering and feature-store design. - Reproducible training: data versioning, seeds, environment pinning. - Train/serve consistency. - Deployment: batch, real-time, shadow. - Monitoring: data drift, prediction drift, performance decay. ## Inputs - The prediction task and the label definition. - The data: its sources, its freshness, and its leakage risks. - Latency and throughput requirements at serving time. ## Outputs - A reproducible training pipeline with a versioned dataset and model. - Features computed by the same code in training and serving. - Drift and performance monitoring with alerts. ## Workflow 1. **Define the label precisely** — Including the time at which it becomes known. A label that is only available thirty days after the prediction cannot be used to evaluate a model deployed today, and this constraint shapes everything. 2. **Check for leakage first** — Any feature computed from information that ...