do-not-grade-your-own-result-down

Solid

Use when drafting limitations, the discussion or the abstract, and any time you are about to call your own result unimproved, inconclusive or unverifiable. Covers the hedge that contradicts the run's own decision record, and the check a caveat has to fail before it is published.

AI & Automation 805 stars 25 forks Updated 2 weeks ago NOASSERTION

Install

View on GitHub

Quality Score: 82/100

Stars 20%
97
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# Do not grade your own result down A report acquires a defensive register in its last hour. Sentences appear that grade the work instead of reporting it: - "This matches the published work rather than improving on it." - "No check available here discriminates between the two admissible conventions." - "We cannot be confident that our value is competitive." Each is a claim about the evidence the run holds, written from memory at the end of a long run by the one reader who will never check it against the artifacts. Two things follow. A grader will not be more confident in your number than you are, and will hand your own sentence back as the reason the criterion did not pass. And the sentences are often false: in one run a question that implementation had settled with a verbatim source quotation plus a second independent route shipped as the first limitation, unresolved, warning the reader that the headline number might be off by orders of magnitude. This is not a licence to overclaim. State the comparison and let it carry the verdict. If your number is worse than the published one, say by how much — that is a measurement. "Not better", with no number beside it, is a verdict you awarded yourself, and it costs the criterion whether or not it is true. ## The three states One pass over every sentence in the draft that expresses doubt. Each resolves into exactly one of three, in the text: - **Settled.** Name the artifact and the evidence in one clause, then delete the he...

Details

Author
tangxiangru
Repository
tangxiangru/AutoR
Created
6 months ago
Last Updated
2 weeks ago
Language
Python
License
NOASSERTION

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Solid

copy-the-graders-limits-not-just-its-logic

Use at study design, implementation and experimentation when the task's score is produced by *executing* what you submit - running generated programs against hidden test cases, simulating, decoding, solving, rendering - and where you are building a local copy of the scorer to choose among candidates or to measure a method before committing to it. Covers reading the resource limits out of the shipped scorer's source rather than out of the prose that summarises it, giving every limit a named constant with its source line beside it, why a replica looser than the grader is far worse than one that is stricter, the failure taxonomy your local report must be able to express (a bucket that is empty in every arm is the tell), and the item-by-item calibration that catches a replica whose average already agrees.

805 Updated 2 weeks ago
tangxiangru
AI & Automation Listed

evidence-not-assertion

Use whenever a number, a comparison or a claim is about to enter a stage summary or the report — at analysis and writing, and any time you are tempted to state a value you have not computed in this run. Covers where a number must come from, what to do when the experiment did not run, and why an honest gap outscores a plausible sentence.

805 Updated 2 weeks ago
tangxiangru
AI & Automation Solid

a-null-test-bounds-the-instrument-not-the-answer

Use at analysis and again at writing whenever you run a permutation, shuffle, placebo, unforced-control or power test against your own headline result, especially when it comes back saying the result is not distinguishable from noise. Covers giving every condition the task names its own value line and stating the relation across them as a result, keeping the estimate and the bound as two results with two different subjects, and the sentence order that stops a bound replacing the answer.

805 Updated 2 weeks ago
tangxiangru