smithers-eval

# smithers eval Run a workflow over a JSON/JSONL eval suite and write a regression report. ## Arguments | Name | Type | Required | Description | |------|------|----------|-------------| | `workflow` | `string` | yes | Workflow ID (from `smithers workflow list`) or path to a .tsx workflow file | ## Options | Flag | Type | Default | Description | |------|------|---------|-------------| | `--cases` | `string` | | JSON or JSONL eval case file | | `--suite` | `string` | | Stable suite ID used in run IDs and report paths | | `--runLabel` | `string` | | Run label appended to eval run IDs; defaults to current UTC timestamp plus a nonce | | `--dryRun` | `boolean` | `false` | Plan the suite without launching runs | | `--concurrency` | `number` | `1` | Number of eval cases to run at once | | `--maxCases` | `number` | | Run only the first N cases | | `--report` | `string` | | Write report JSON to this path | | `--force` | `boolean` | `false` | Overwrite an existing eval report | | `--includeOutput` | `boolean` | `true` | Include workflow outputs in the report | | `--maxConcurrency` | `number` | | Per-workflow max task concurrency | | `--root` | `string` | | Tool sandbox root directory | | `--log` | `boolean` | `true` | Enable NDJSON event log file output | | `--logDir` | `string` | | NDJSON event logs directory | | `--allowNetwork` | `boolean` | `false` | Allow bash tool network requests | | `--maxOutputBytes` | `number` | | Max bytes a single tool call can return | | `--to...