detecting-data-and-model-poisoning

# Detecting Data and Model Poisoning > **Authorized-use-only notice:** This skill includes routines that craft poisoned samples and backdoor triggers for *defensive validation*. Generate and use poisoned data and backdoored models only in isolated test environments you control. Never deploy a backdoored model or distribute poisoned datasets. ## Overview Data poisoning and model backdooring attack the *integrity* of an ML system at training time rather than at inference. In **data poisoning** (MITRE ATLAS **AML.T0020 Poison Training Data**), an adversary injects manipulated samples into the training, fine-tuning, or RAG corpus so the resulting model misbehaves — degraded accuracy, targeted misclassification, or an attacker-chosen bias. In **model backdooring** (MITRE ATLAS **AML.T0018 Backdoor ML Model**), the model behaves normally on clean inputs but produces an attacker-chosen output whenever a hidden *trigger* (a pixel patch, a rare token, a phrase) is present. Both are amplified by **ML supply-chain compromise (AML.T0010)**: poisoned public datasets, trojaned pre-trained weights downloaded from a hub, or a malicious model serialization. This is OWASP **LLM04:2025 Data and Model Poisoning**. Detection spans the pipeline. On the *data* side: provenance and integrity checks, statistical outlier and label-flip detection, and de-duplication of suspiciously near-identical samples. On the *model* side: activation-clustering and spectral-signature analysis (which exploit the ...