a-combination-is-not-the-candidate-set

Solid

Use once more than one trained artifact exists on disk -- two checkpoints, two seeds, two architectures, a continuation run -- and something is deciding which of them, or which combination of them, writes the predictions file. Covers the ballot that lists every artifact as a submission on its own before any blend, re-running it whenever a training job finishes, the known-bad canary that tests the objective, and persisting a rejected candidate's predictions.

AI & Automation 805 stars 25 forks Updated 2 weeks ago NOASSERTION

Install

View on GitHub

Quality Score: 82/100

Stars 20%
97
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Every artifact you built is a candidate submission; the combinations are extras A run that trains several models grows a write gate: an incumbent, a stream of challengers, a bar, a decision. The gate is the right instinct. It fails in a way that looks like discipline from the inside. `a-scoreable-file-in-the-first-hour` owns the incumbent-versus-challenger loop and `select-on-the-file-you-would-ship` owns the number it compares. This owns *which candidates enter it*. One run's gate ran at every new checkpoint and held every time. Its final cycle priced 3,184 weighted combinations of six arms, picked a four-way blend, and refused it. Every decision was correct on its own terms. The single best artifact in the workspace -- one checkpoint, alone, with the task's own post-processing on top -- was never entered as a candidate in its own right, and it scores 2.2 points above what shipped and above the scaffold-free control run of the same task on the same hardware. The gate was choosing carefully among things that all shared a defect. ## 1. Keep a ballot, and put the singletons on it first N artifacts means the candidate set *starts* as N singletons. Combinations are additions to it, never a replacement for it. ```python cands = {tag: single(tag) for tag in artifacts} # first, unconditionally cands |= {name: combo(w) for name, w in weight_grid} # then the search ``` Write the result as a file, one row per candidate -- tag, path to its persisted per-row scores, score...

Details

Author
tangxiangru
Repository
tangxiangru/AutoR
Created
6 months ago
Last Updated
2 weeks ago
Language
Python
License
NOASSERTION

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category