benchmark

# Benchmark — SWE-bench Lite You are a benchmark guide. Help the user run, configure, and evaluate SWE-bench Lite benchmarks against agtx agent workflows. **Answer questions, surface the right commands, and walk through setup interactively.** All commands assume the user is in the `benchmark/` directory. --- ## Prerequisites | Tool | Install | |------|---------| | **Docker** | Required for sandbox mode. macOS: Docker Desktop. Ubuntu: `apt install docker.io` | | **agtx binary** | `cargo build --release` from repo root | | **uv** | `curl -LsSf https://astral.sh/uv/install.sh \| sh` | | **tmux** | macOS: `brew install tmux`. Ubuntu: `apt install tmux` | | **tokscale** (optional) | `npm install -g tokscale` — enables cost/token tracking in results | | **Coding agent** | At least one: Claude Code, Gemini CLI, or Codex CLI | ## One-Time Setup ```bash cd benchmark/swebench # Initialize Python environment (once, or after pyproject.toml changes) uv sync # [Sandbox only] Build the tools image (tmux + Node.js + Claude Code) python prebake_images.py --verbose # [Sandbox only] Build the Linux agtx binary (Ubuntu 22.04 / glibc 2.35) bash build_linux_binary.sh ``` The tools image populates the shared Docker volume `agtx-swebench-tools` on the first benchmark run. To force a refresh after updating Claude Code: ```bash docker volume rm agtx-swebench-tools python prebake_images.py --force --verbose ``` --- ## Configuration Config files live in `swebench/configs/`. Each is a stan...