nasde-dev

Solid

Internal skill for developing and maintaining nasde-toolkit itself. Use this skill when: - Making changes to nasde-toolkit source code (CLI, runner, evaluator, config, agents) - Refactoring or adding features to the toolkit - Fixing bugs in the evaluation pipeline - Updating dependencies or integration points (Harbor, Opik, `claude` / `codex` CLI subprocess backends) This skill defines the verification protocol that must be followed after any significant change.

AI & Automation 11 stars 0 forks Updated today MIT

Install

View on GitHub

Quality Score: 79/100

Stars 20%
36
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# NASDE Toolkit Development Internal guidelines for developing nasde-toolkit itself. > **Important context:** `nasde-toolkit/` is the current production toolkit. The repository also contains `SDLC/evals/` which is the **legacy predecessor** — the original research/prototype that has been migrated into nasde-toolkit. Do NOT modify `SDLC/evals/` for new work. It will be removed once migration is confirmed complete. ## Post-change verification protocol After any significant change to the toolkit (new features, refactors, dependency updates, renamed identifiers), run the full verification pipeline before considering the work complete. ### 1. Quality gates (must pass before any commit/PR) These are the same checks that run on CI. **Always run locally before claiming code is ready:** ```bash # Lint uv run ruff check src/ tests/ # Format uv run ruff format --check src/ tests/ # Type check uv run mypy src/nasde_toolkit/ # Unit tests uv run pytest ``` All four must pass. If `ruff format` fails, run `uv run ruff format src/ tests/` to auto-fix. After pushing, verify CI passes on the PR with `gh pr checks <PR_NUMBER>`. ### 1b. Static checks ```bash # No stale references to old names or patterns grep -r "sdlc.eval\|sdlc-eval\|sdlc_eval" src/ docs/ CLAUDE.md README.md .claude/skills/ pyproject.toml --include="*.py" --include="*.md" --include="*.toml" --include="*.json" | grep -v __pycache__ # Package installs cleanly uv sync ``` ### 1c. Documentation and skills consistency...

Details

Author
NoesisVision
Repository
NoesisVision/nasde-toolkit
Created
4 months ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

nasde-benchmark-from-public-repos

Build diverse benchmark task suites from public GitHub repositories for testing universal skills. Use this skill when the user wants to: - Create a benchmark that spans multiple public repositories and languages - Test a universal skill (refactoring, test writing, code review, etc.) across diverse codebases - Curate a representative set of repos and tasks for cross-codebase validation - Build an evaluation suite for a skill that should work in any repository Even if the user doesn't say "benchmark" — if they're building a skill meant to work everywhere and want to validate it across many different projects, this skill applies.

11 Updated today
NoesisVision
AI & Automation Solid

nasde-benchmark-runner

Run coding agent benchmarks and verify results with nasde. Use this skill when the user wants to: - Run a benchmark (all tasks, single task, specific variant) - Re-run assessment evaluation on existing trial results - Check or verify results in Opik (traces, feedback scores, experiments) - Troubleshoot a failed benchmark run - View or compare trial results Even if the user doesn't say "benchmark" — if they're talking about running evaluations, checking scores, or analyzing agent performance, this skill applies. After every run that uses --with-opik, ALWAYS verify results via Opik REST API — don't wait for the user to ask.

11 Updated today
NoesisVision
Code & Development Solid

nasde-benchmark-calibration

Calibrate assessment rubrics by reviewing agent work in GitHub/GitLab PRs and feeding human comments back into the rubric. Use this skill when the user wants to: - Calibrate, tune, or sanity-check assessment criteria / dimensions of a benchmark - Review trial diffs alongside the LLM-as-a-Judge scores in a PR/MR - Investigate why judge scores feel off, too harsh, too lenient, or misaligned with how a human would grade the code - Pull review comments back from PRs/MRs and turn them into concrete rubric edits Even if the user doesn't say "calibrate" — if they're worried the LLM judge's scores diverge from human judgment, or want to align scores with a real developer's opinion before freezing a benchmark, this skill applies.

11 Updated today
NoesisVision