crossval-harness

Solid

Orchestrate a static + dynamic, exploit-validated red-team of an AI agent — read the source to find candidate vulnerable paths, then run the dynamic skills to confirm or refute each one empirically. The arbiter of truth is whether the exploit works, not a model vote. Authorized testing of agents you own or are permitted to test.

AI & Automation 1 stars 0 forks Updated 1 weeks ago MIT

Install

View on GitHub

Quality Score: 78/100

Stars 20%
10
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Cross-validation harness · static + dynamic ## What this is The orchestration skill. It runs the two halves and correlates them so you get *proven* findings and root cause — not a wall of static maybes, and not a break with no explanation. ## Authorization & safety — read first Follow every rule in [`../README.md`](../README.md). Authorized target, disposable environment, no real credentials, harmless markers, no destruction, responsible disclosure. Otherwise **stop**. ## Procedure 1. **Static triage — where to look.** Read the agent's source and flag candidate paths: untrusted content reaching a dangerous sink, a signature-only guard, a missing check, an un-sandboxed exec. Map each candidate to a class (RT-1…RT-9). Tooling: `python3 scanner/static_scan.py --source /path/to/agent`. A candidate is **not** a finding. 2. **Dynamic confirmation — what's real.** For each candidate, run the matching skill (`rt1-prompt-injection`, `rt6-memory-poisoning`, `rt3-sandbox-escape`, `rt4-action-gating`) against the running agent. A finding is **confirmed** only when the exploit actually works. 3. **Correlate.** - **Confirmed:** trace the working exploit back to the exact code path → root cause, blast radius, and where to fix. - **Refuted:** a static candidate the dynamic pass couldn't exploit — kill it out loud. A false positive dies to a real attack, not a model vote. 4. **Grade & report.** Each finding carries both the code path and a working exploit, ...

Details

Author
William2333ZZ
Repository
William2333ZZ/trustshell
Created
1 weeks ago
Last Updated
1 weeks ago
Language
Python
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

redteam-an-agent

The end-to-end methodology for red-teaming a specific AI agent — adaptively, exploit-validated, and honestly. Read THIS target's own code, stand up a disposable harness, and prove or refute each weakness through a real attacker-reachable entry point. This is the orchestration + discipline that makes a finding credible, not a list of payloads. Authorized testing of agents you own or are permitted to test.

1 Updated 1 weeks ago
William2333ZZ
Code & Development Listed

harness-health-audit

Audit the health of an AI-coding harness — the CLAUDE.md / AGENTS.md files, rules, skills, agents, hooks, and commands that steer an agent in a repository — across four questions a linter cannot answer: is it TRUE (does it match the real code?), is it USED (do skills actually trigger and does the loop actually verify?), is it AFFORDABLE (is it inside the budget where instructions are still followed?), and is it SAFE (is a live, accurate instruction still a good idea — permission bypasses, unpinned remote scripts, untrusted content flowing in as instructions? — safety findings are flagged for the user's decision, never auto-removed). Then surgically correct or remove only what is proven wrong or dead. Use whenever the steering layer may have rotted or may not be working: "하네스 점검해줘", "하네스 드리프트 감사", "CLAUDE.md가 실제랑 맞는지 봐줘", "룰/스킬이 최신인지 확인해줘", "스킬이 제대로 활용되는지 봐줘", "루프 엔지니어링 잘 되고 있는지 검토해줘", "죽은 훅/커맨드 정리해줘", "하네스 안전한지 점검해줘", "audit my harness", "check the rules still match the real stack", "are my skills actually be

3 Updated today
uzysjung
AI & Automation Listed

dynamic-verification

Confirm or refute a vulnerability candidate against a live target with a deterministic probe (the agent runs the probe, respecting authorization and scope) — Applies to: when a static/SAST or LLM review flags a possible injection or SSRF; when triaging a finding before opening a bug or shipping a fix; when an authorized dynamic test against a running app is available; when a verification result turns out wrong and the PoC or skill needs fixing

3 Updated 3 days ago
ShieldNet-360