generate-and-gradelisted
Install: claude install-skill seldonframe/seldonframe
# generate-and-grade
A deterministic regression net for the agent generator. It runs a fixed prompt
set through the generator's **no-LLM** path — `assembleAgentBundle(heuristicIntent(sentence))`
— and asserts each sentence still yields a SANE agent (right trigger kind, expected
tool bindings, expected channel, correct skill template). No live LLM key needed;
it is reproducible in CI and locally.
## When to use
After ANY change to:
- `packages/crm/src/lib/agents/generate/**` (parse-intent.ts, agent-bundle.ts,
bind-tools.ts, tool-catalog.ts)
- the trigger model (`src/lib/agents/triggers/agent-trigger.ts`)
- the starter templates (`src/lib/agent-templates/starter-pack.ts`)
## Run it
From `packages/crm`:
```
node --import tsx --test tests/regression/generator-prompts.spec.ts
```
Expect: **all cases pass** (`fail 0`). The baseline is green; any red is a regression.
## On failure — STOP and report
If any case fails, the generator regressed. Do NOT proceed with the change. Read
the failure message — it names the exact sentence and every mismatch on its own
line, e.g.:
```
Generator regression for sentence: "Post a weekly Instagram highlight of our 5-star reviews"
• trigger.kind: expected "schedule", got "event"
• connectors: missing required tool id(s) [postiz] — bound: [none]
��� skill: customSkillMd contains forbidden signature(s) ["You are the review-requester"] — wrong skill template was used
```
Report to the user: the sentence + which axis broke (trigger / t