weak-agent-test

Solid

Run the weak-agent adversarial test harness against docx-cli. Spawns weak exercise agents (Haiku by default, Sonnet to probe, or a local agent harness's pre-produced runs) to perform real document tasks over six scenarios — five editing (MNDA form-fill + font fidelity, invoice table-edit/restructure + logo replace, résumé styling, contract redlining + commenting, contract finalize via accept/reject + comment reply/resolve) and one authoring (T. S. Eliot poetry journal: multi-column, verse, footnotes, links, figure) — renders every result with Word, has opus judge them against ground-truth rubrics, measures each exercise's tool economy, token cost, wall-clock, and correctness (from transcripts for Claude, the exercise.json ledger for the local harness), and synthesizes a prioritized ergonomics report. Use when the user says 'adversarial review', 'test docx-cli with weak agents', 'run the haiku harness', 'weak agent test', or wants to re-run yesterday's adversarial process.

Data & Documents 154 stars 8 forks Updated 1 weeks ago MIT

Install

View on GitHub

Quality Score: 88/100

Stars 20%
73
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Adversarial review — weak-agent harness for docx-cli This harness answers one question: **can weak agents actually use docx-cli to get real work done, and what should we fix first?** It runs the `weak-agent-test` workflow (`.claude/workflows/weak-agent-test.js`), which fans out one weak exercise agent per scenario (Haiku by default — swappable to Sonnet via `args.model`), renders every output with Microsoft Word, grades each against ground-truth criteria with an **opus** judge, and has **opus** synthesize a prioritized improvement report. Exercise agents do NOT self-report tool counts — every tool-economy and token number is **measured** after the run (agents under-count their own calls ~2×, so self-reports were dropped): from the agent transcripts for the Claude arms, from each scenario's `exercise.json` ledger for the local arm. Both roll up into the same Run-metrics table (tokens, wall-clock, tool split, correctness) via `exercise-metrics.ts`. The test corpus is **bundled with this skill** under `scenarios/`, one folder per scenario, named after its key (`scenarios/mnda/`, `scenarios/invoice/`, …). Each scenario folder is self-describing and holds everything that scenario needs: - `task.md` — the AGENT-FACING request, written as a human delegating the work: the goal, the data, the intent — and **no tool vocabulary** (no `docx` commands, locators, or OOXML terms), because discovering which features deliver the outcome is part of what's measured, - `criteria.md` —...

Details

Author
kklimuk
Repository
kklimuk/docx-cli
Created
2 months ago
Last Updated
1 weeks ago
Language
TypeScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

Code & Development Solid

agent-benchmark

Self-benchmark: YOU write the code, adversarial reviews it (multi-provider), you fix, you write tests, adversarial reviews tests, you fix. Measures YOUR quality as an agent. Run in different models (Opus, Sonnet, Haiku) and compare results.

6 Updated today
greglas75
Data & Documents Solid

word-docx

Unified skill for Microsoft Word .docx workflows: extracting comments or tracked changes, summarizing reviewer feedback, generating response-to-comments documents, building Word memos or reports, applying tracked edits, auditing OOXML internals, validating and repairing documents, accepting revisions, adding or resolving comments, converting legacy .doc files, unpacking or repacking .docx files, simplifying redline markup, rendering .docx to images, and building Word documents with tables of contents, multi-column layouts, or page-numbered headers and footers. Use whenever a task involves reading, reviewing, building, editing, or analyzing .docx files. Prefer this skill over ad-hoc python-docx or docx2python usage. Route PDFs, spreadsheets, presentations, Google Docs, and LaTeX-to-Word conversion pipelines to their dedicated skills.

24 Updated 4 days ago
kennethkhoocy
Testing & QA Listed

testing-skills-with-subagents

TDD applied to skill/process documents — pressure-test a discipline-enforcing skill with subagents BEFORE deploying it, by running realistic multi-pressure scenarios without the skill (RED, capture rationalizations verbatim), with it (GREEN), then closing each loophole (REFACTOR) until agents comply under maximum pressure. Use when creating or editing any gate/discipline skill in this repo — especially before /ms.sync broadcasts the edit to every registered project repo. Skip for pure reference skills with no rules to violate.

0 Updated 3 days ago
beomeodev