eval-benchlisted

# Benchmarking a model against this server ## The grid, and what bites while running it `docs/local-model-eval.md` is the result; this is what bites while running it again. The grid is four scripts — the ollama driver, the Claude Code driver, the Foundation Models driver, and the matrix runner that spawns one of the three per cell and grades the log afterwards. **The `fm` backend varies none of the grid's three axes, and that is its result rather than a gap.** Apple's on-device model has one model with no tag to choose, a window fixed at whatever the OS enforces, and `reasoning: false` — so a plan naming another model, asking for a `context`, or setting `think` is **refused** at validation rather than run. Each of those would otherwise key a cell against a record that carries something else (`num_ctx: null`, one fixed model name), and a cell nothing can match re-runs on every invocation while the plan goes on claiming the axis. Only `--tools crash` fits the window at all, so the surface axis collapses to one cell too. Read its row on its own and never fold it into an aggregate with the ollama cells; `docs/apple-foundation-models.md` is the write-up and carries the measurements. **Record what the runtime *served*, not what you asked for.** `num_ctx` on a request does not shrink an instance ollama already holds: with a 32,768 instance loaded, cells asking for 8,192 are served 32,768 and look perfectly healthy — a 17,300-token prompt "fitting" in 8k, which is the result the