harness-canary-authoring

Solid

Picking and asserting harden-harness canaries

AI & Automation 69 stars 13 forks Updated today MIT

Install

View on GitHub

Quality Score: 81/100

Stars 20%
61
Recency 20%
100
Frontmatter 20%
40
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Harness Canary Authoring Use this when adding a new canary to `tests/harden.rs`. Canaries catch real regressions; bad canaries become flaky CI noise. ## Pick a good canary A canary symbol or query must be: - **Call-site-dense in the target repo** — at least 50× the threshold. If you assert `>= 200` hits, pick a callee with ≥ 1000 actual call sites in the repo. Headroom absorbs upstream churn. - **Stable across releases** — pick a fundamental API (`tokio::spawn`, Django's `get`), not a freshly renamed one. - **Unambiguous by name alone** — basemind's reference search is name-only. `get` is fine inside Django (many call sites, all the same semantic). Inside React, `get` would match too many unrelated callees; pick `useState` instead. - **Cheap to scan** — `scan_cap = limit * 8` bounds work, but a hit-dense canary still has the lowest overhead. ## Assertion shape - Always use lower bounds: `assert!(hits >= N, "expected >= N, got {hits}")`. - Never assert equality on counts — upstream repo evolution breaks equality assertions silently. - Capture the canary count in the per-repo metrics struct so regressions are visible in the JSON log even when the assertion passes. ## Steps 1. Pick the target repo + symbol per the criteria above. 2. Confirm the count locally: clone the repo, run `basemind scan`, call `find_references` via the MCP, note the actual count. 3. Set the threshold to `actual / 2` rounded down — survives ~50% churn. 4. Add to `tests/harden.rs`: - The canary...

Details

Author
Goldziher
Repository
Goldziher/basemind
Created
2 years ago
Last Updated
today
Language
Rust
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category