← ClaudeAtlas

tasklisted

Turn a natural-language description into coder-eval task YAML — minimal prompts, weighted criteria that check output content, validated with `coder-eval plan`. Use when the user wants to write, add, or generate an evaluation task.
UiPath/coder_eval · ★ 122 · AI & Automation · score 80
Install: claude install-skill UiPath/coder_eval
# Author a coder-eval task You are writing coder-eval task YAML. The user's request is: `$ARGUMENTS` If `$ARGUMENTS` is empty, ask what the task should test. Do not invent a subject. Good tasks use simple prompts: state the goal and the expected output, then let the agent work out the approach. A single request can produce **several** task files — "create tasks for all the registry subcommands" means one task per subcommand. ## Step 1 — Understand the request, and check the CLI is there Run `coder-eval --version` first. Steps 6 and 7 both shell out to it, and finding that out *after* writing several task files means the user gets a bare `command not found` with nothing to act on. Installing this plugin did not install the CLI. If it is missing, follow `${CLAUDE_PLUGIN_ROOT}/reference/cli-setup.md`: offer the install, **ask before running it**, and confirm with `coder-eval --version` afterwards. Never install unprompted, and do not write any task files if the user declines. That reference also covers the other half of the version check — whether this project pins a coder-eval version, and what to do when the installed one does not match it. Then establish: - **What is being tested** — which tool, SDK, CLI, skill, or capability? - **How many tasks** — one operation, or several? - **Difficulty** — smoke, basic, or intermediate? - **Dependencies** — network, packages, starter files, external services? State any assumptions you make rather than silently picking. ## Step