add-the-baseline-back-on-the-split-you-cannot-score
SolidUse at study design, implementation and every write thereafter, whenever the value you submit is assembled from parts - a fitted baseline plus a model's residual, a level plus a shape, a de-trended prediction that has to be re-trended, any inverse transform - and the validation arrays and the graded arrays are produced by separate calls. Covers assembling every split through one function, using the baseline you already fitted as a label-free reference vector on the graded split, why row count, header, dtype and finiteness cannot see this class of error, and putting the gate inside the writer rather than in a script somebody has to remember to run.
Install
Quality Score: 82/100
Skill Content
Details
- Author
- tangxiangru
- Repository
- tangxiangru/AutoR
- Created
- 6 months ago
- Last Updated
- 2 weeks ago
- Language
- Python
- License
- NOASSERTION
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
a-scoreable-file-in-the-first-hour
Use at the first stage of a run whose deliverable is a predictions file, and again at every stage when one still does not exist. Covers why a trivial submission written early dominates a good one written late, what the first version should contain, and how to improve it in place without ever leaving it invalid.
chemistry-reproduce-the-scoring-path-before-you-replace-it
Use at implementation, experimentation and analysis when you are reproducing a published benchmark number and the source's scoring path is one you can read — which rows are scored, in what order, how many the loader drops, which epoch is reported, how tasks are pooled, over how many seeds. Covers implementing that path exactly before improving it, the one-row-per-step ladder from the published rule down to your own honest estimate, and why one un-replicated step makes the reproduction gap you report uninterpretable.
calibrate-the-level-on-the-window-you-cannot-score
Use at implementation and afterwards whenever the rows you will be scored on lie outside every window you can check against truth — a forecast horizon that starts where the supplied history ends, a later time period, a different site, batch or cohort, a test split whose label column has been removed — and your only bias check was run on a backtest fold or a random validation split. Covers why "my predictions are unbiased" is a statement about the folds and not about the graded rows, how to measure the overall level of your predictions on the graded rows with no labels at all, why the level ratio is only the alarm and a metric scan is the number, what to do when the two windows disagree, and when a low forecast is correct rather than broken.