paper-claim-audit

Solid

Zero-context verification that every number, comparison, and scope claim in the paper matches raw result files. Uses a fresh cross-model reviewer with NO prior context to prevent confirmation bias. Use when user says "审查论文数据", "check paper claims", "verify numbers", "论文数字核对", or before submission to ensure paper-to-evidence fidelity.

AI & Automation 11,152 stars 1050 forks Updated today MIT

Install

View on GitHub

Quality Score: 96/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Paper Claim Audit: Zero-Context Evidence Verification > 🔒 **Do not wrap this skill in `/loop`, `/schedule`, or `CronCreate`.** It is > verdict-bearing — it judges paper-to-evidence fidelity with a deliberately > zero-context fresh reviewer. Re-firing that verdict on a wall-clock timer adds > no new signal (it changes only when the *paper or results* change). Schedule > the *external wait that precedes it* — paper draft ready → then audit > **once**. See > [`shared-references/external-cadence.md`](../shared-references/external-cadence.md). Verify that every claim in the paper matches raw evidence for: **$ARGUMENTS** ## Why This Exists The executor writes experiments AND writes the paper. It "knows" what the results should be. This creates confirmation bias: - Rounding 84.7% up to 85.3% - Reporting best seed instead of average - Citing metrics from a different experiment config - Claiming "improves by 15%" when the delta is actually 12.8% A **fresh reviewer with zero prior context** catches these because it has no expectations — it just compares paper text vs raw files. ## How This Differs From Other Audit Skills | Skill | Question it answers | |-------|-------------------| | `/experiment-audit` | Is the experiment code honest? (fake GT, normalization fraud) | | `/result-to-claim` | Does the data scientifically support this claim? | | **`/paper-claim-audit`** | **Does the paper report the data truthfully and precisely?** | ## Core Principle **Zero-context, fresh rev...

Details

Author
wanshuiyin
Repository
wanshuiyin/Auto-claude-code-research-in-sleep
Created
2 months ago
Last Updated
today
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

citation-audit

Zero-context verification that every bibliographic entry in the paper is real, correctly attributed, and used in a context the cited paper actually supports. Uses a fresh cross-model reviewer with web/DBLP/arXiv lookup to catch hallucinated authors, wrong years, fabricated venues, version mismatches, and wrong-context citations (cite present but the cited paper does not establish the claim). Use when user says "审查引用", "check citations", "citation audit", "verify references", "引用核对", or before submission to ensure bibliography integrity.

11,152 Updated today
wanshuiyin
AI & Automation Listed

verify

Paper-vs-code consistency audit. After research:scientist implements a method from a paper, verify the implementation matches paper claims across five dimensions — formula matching [F], hyperparameter parity [H], eval protocol [E], notation consistency [N], and citation chain [C]. Reads paper (PDF path / arXiv URL / pasted text), maps claims to codebase, emits verification table with match status and severity.

17 Updated 2 days ago
Borda
AI & Automation Solid

experiment-audit

Audit experiment integrity before claiming results. Uses cross-model review (external reviewer backend) to check for fake ground truth, score normalization fraud, phantom results, and insufficient scope. Use when user says "审计实验", "check experiment integrity", "audit results", "实验诚实度", or after experiments complete before writing claims.

11,152 Updated today
wanshuiyin
AI & Automation Listed

paper-audit

Use this skill when the user wants to audit their paper pipeline, run a pre-submission check, or verify quality before submitting. Triggers include: "audit pipeline", "check everything", "pre-submission check", "quality check", "verify pipeline", "final check". Also use as the mandatory final step before any submission (Rule 16).

2 Updated 4 days ago
charlotte-12s
AI & Automation Listed

ref-verify

Prevents citation hallucination in academic writing. Invoke when: finding papers to support a specific claim; verifying/checking/auditing existing citations or DOIs; confirming whether a paper actually says what the user claims it says ('is that what the paper says?', 'did they actually show X?'); adding a citation by describing a paper ('add a citation for the paper where X'); running a pre-submission reference sweep. Do NOT invoke for: formatting references in APA/IEEE style, general topic explanations, citation style questions, or prose editing. Selects Quick Screen (seconds per paper) or Full Audit (abstract fetch + claim check) automatically.

0 Updated today
moonweave