a-throughput-number-belongs-to-the-runtime-not-the-model

# You timed a configuration. Do not record it as a verdict on the component Early in a run you pick a component — a pretrained checkpoint, a solver, a library — load it the first way you find, run one job, and read a number. If the number is bad you write "too slow" and fall back. What you measured is not the component. It is a quadruple: > **component × runtime × workload × machine under this load** Exactly one of those four costs you capability when you change it, and it is the component. It is also the one that gets changed, because *this model is too slow* is a sentence and *the wheel I installed does no vectorised arithmetic on this CPU* is a diagnosis. ## The failure shape, which is not the one you expect The instructive version is not an agent that fails to diagnose. Here is one that diagnosed it correctly and lost the component anyway. It wanted a sparse mixture-of-experts checkpoint for a reason it wrote down: about 3B of its ~30B parameters are active per token, so a large model is affordable on a CPU. It fetched the first distribution format it found — a quantised file for a CPU *generation* server — and measured **78 tokens in 24 seconds**, roughly 20 GFLOP/s against the ~1163 GFLOP/s the same box reached with a plain torch fp32 matmul. Then it got the diagnosis exactly right, in one sentence: no bf16 instruction on this CPU, load average 13.96 on a shared 16-core box, and a generic wheel. It switched runtime, to torch. In the same minute it started a bac...