quant-data-preplisted

# quant-data-prep — Data Preparation & Quality Audit ## Objective Load a research dataset, run a systematic quality audit, and produce a clean, well-labelled dataset with a `column_metadata.json` file. All downstream analysis skills depend on this output. The skill never makes destructive changes silently — every flagged issue is surfaced to the user via `AskUserQuestion` with a recommended action. --- ## Pre-flight **Required input from user:** - Path to data file (`.csv`, `.tsv`, `.xlsx`, or `.xls`) - Nothing else is required upfront — the skill will ask for everything else interactively **Minimum requirements:** - At least 2 columns, at least 10 rows (warn below 30 rows — most analyses will have low power) **Supported formats:** - CSV / TSV — via `pandas.read_csv()` - Excel (.xlsx, .xls) — via `pandas.read_excel()` --- ## Pipeline ### Step 1 — Load data ```python # See references/pipeline.py → load_data() ``` - Auto-detect delimiter for CSV/TSV - Read all columns as-is (do not coerce types yet) - Report: n rows, n columns, file size, preview of first 5 rows **Tell the user:** "I've loaded your file. Here's a quick preview: [table]. Now I'll run a full data quality audit." --- ### Step 2 — Data Quality Audit (always runs) Run all checks below. Collect results into a quality report. Present the full report before asking any questions. #### 2a. Structural checks | Check | 🟢 | 🟡 | 🔴 | |---|---|---|---| | Duplicate rows | 0 | 1–2% | >2% | | Column names | cle