nasde-benchmark-creatorlisted
Install: claude install-skill NoesisVision/nasde-toolkit
# NASDE Benchmark Creator
Create and configure coding agent benchmarks for evaluation with `nasde`. A benchmark is a set of coding tasks that AI agents solve inside isolated Docker containers, scored both by functional tests (pass/fail) and by an LLM-as-a-Judge architecture assessment.
## Critical: line endings on Windows (read this first)
Benchmark scripts execute inside **Linux** sandboxes (Docker, Daytona). If `tests/test.sh`, `solution/solve.sh`, or `environment/Dockerfile` are checked out with **CRLF** line endings (the Windows git default when `core.autocrlf=true` and there is no `.gitattributes`), every trial fails immediately with:
```
bash: line 1: /tests/test.sh: cannot execute: required file not found
```
…because the kernel reads the shebang as `#!/bin/bash\r` and tries to execute a non-existent `/bin/bash\r`. The agent finishes its work, but the verifier never runs and Harbor reports `RewardFileNotFoundError`.
**Mitigation (always do this for a new benchmark — `nasde init` does it for you, but verify):**
1. The benchmark repo MUST have a `.gitattributes` file enforcing LF for shell scripts and Dockerfiles. The minimum content:
```gitattributes
* text=auto eol=lf
*.sh text eol=lf
*.bash text eol=lf
Dockerfile text eol=lf
*.dockerfile text eol=lf
docker-compose.yaml text eol=lf
docker-compose.yml text eol=lf
*.ps1 text eol=crlf
*.bat text eol=crlf
*.cmd text eol=crlf
```
`nasde init` wri