synthetic-datalisted

**CONSUMES**: User dataset-curation task + raw corpus / seed tasks / fine-tune data description + target model + optional existing pipeline configs **PRODUCES**: Applied data-curation judgment rules + filtering thresholds + dedup architecture choice + synthetic-generation pipeline + preference-pair config + chat-template alignment + contamination audit results # Synthetic Data & Fine-Tune Dataset Curation Capability Pack **Version**: 0.1.0 **Compatibility**: Claude Code (Phase 1); Codex / Cursor / Gemini in Phase 3 **License**: Apache 2.0 --- ## What This Pack Does AI agents build fine-tune datasets by dumping a model's raw generations into a JSONL and training on it. They skip deduplication, so the model memorizes repeated documents. They prompt a single model to generate instructions with no diversity controls and no ROUGE-L filtering, so the set collapses into near-duplicates. They evaluate the fine-tuned model on public benchmarks that leaked into the data and celebrate inflated scores. They mis-map chat templates and train on pad tokens. This pack embeds the judgment rules that data-curation engineers apply automatically — rules from the Self-Instruct and Evol-Instruct/WizardLM papers, LSHBloom-scale dedup architectures, distilabel pipeline practice, Axolotl/Unsloth fine-tuning docs, and the ConTAM/CoDeC contamination-detection literature. **Pack = data-curation judgment. Your workflow system = process constraints. No overlap.** --- ## Cross-Cutting Rule: Decont