vector-forge

Solid

Mutation-driven test vector generation. Finds implementations of a cryptographic algorithm or protocol, runs mutation testing to identify escaped mutants, then generates new test vectors that deliberately exercise the uncovered code paths. Compares before/after mutation kill rates to prove vector effectiveness. Use when generating cryptographic test vectors, measuring Wycheproof coverage gaps, finding escaped mutants via mutation testing, creating cross-implementation test suites, or improving test vector coverage for crypto primitives.

AI & Automation 5,501 stars 484 forks Updated 4 days ago CC-BY-SA-4.0

Install

View on GitHub

Quality Score: 90/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Vector Forge Uses mutation testing to systematically identify gaps in test vector coverage, then generates new test vectors that close those gaps. Measures effectiveness by comparing mutation kill rates before and after. ## When to Use - Generating test vectors for cryptographic algorithms or protocols - Evaluating how well existing test vectors cover an implementation - Finding implementation code paths that no test vector exercises - Creating Wycheproof-style cross-implementation test vectors - Measuring the concrete coverage value of a test vector suite ## When NOT to Use - No implementations exist yet (need code to mutate) - Single trivial implementation with no edge cases - Testing application logic rather than algorithm implementations - The algorithm has no public test vectors to compare against ## Prerequisites - **trailmark** installed — if `uv run trailmark` fails, run: ```bash uv pip install trailmark ``` - At least one implementation of the target algorithm in a language with mutation testing support - A test harness that consumes test vectors and exercises the implementation - A mutation testing framework for the target language --- ## Rationalizations to Reject | Rationalization | Why It's Wrong | Required Action | |-----------------|----------------|-----------------| | "We have enough test vectors" | Mutation testing proves otherwise | Run the baseline first | | "The implementation's own tests are sufficient" | Own tests often share blind...

Details

Author
trailofbits
Repository
trailofbits/skills
Created
4 months ago
Last Updated
4 days ago
Language
Python
License
CC-BY-SA-4.0

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

genotoxic

Graph-informed mutation testing triage. Parses codebases with Trailmark, runs mutation testing and necessist, then uses survived mutants, unnecessary test statements, and call graph data to identify false positives, missing test coverage, and fuzzing targets. Use when triaging survived mutants, analyzing mutation testing results, identifying test gaps, finding fuzzing targets from weak tests, running mutation frameworks (including circomvent and cairo-mutants), or using necessist.

1 Updated 4 days ago
kevinvwong
Testing & QA Solid

genotoxic

Graph-informed mutation testing triage. Parses codebases with Trailmark, runs mutation testing and necessist, then uses survived mutants, unnecessary test statements, and call graph data to identify false positives, missing test coverage, and fuzzing targets. Use when triaging survived mutants, analyzing mutation testing results, identifying test gaps, finding fuzzing targets from weak tests, running mutation frameworks (including circomvent and cairo-mutants), or using necessist.

5,501 Updated 4 days ago
trailofbits
Testing & QA Listed

atdd-mutate

Use to add a third validation layer to the ATDD workflow — after acceptance tests verify WHAT and unit tests verify HOW, mutation testing verifies the tests actually catch bugs. Triggers — "/mutate", "/kill-mutants", "run mutation testing", "mutate my code", "kill mutants", "check test quality", "find surviving mutants", "run stryker", "run mutmut", "run pitest", "are my tests catching bugs".

100 Updated 3 days ago
swingerman
Testing & QA Solid

mutation-testing

Validate test effectiveness with mutation testing using Stryker (TypeScript/JavaScript with Vitest or bun test via @hughescr/stryker-bun-runner) and mutmut (Python). Find weak tests that pass despite code mutations. Use to improve test quality.

162 Updated 2 weeks ago
secondsky
Testing & QA Solid

wycheproof

Wycheproof provides test vectors for validating cryptographic implementations. Use when testing crypto code for known attacks and edge cases.

5,501 Updated 4 days ago
trailofbits