arxiv-source-first-paper-ingest

# arXiv source-first paper ingest > **Self-Evolving skill.** If a step here fails or the tooling drifts, fix this file in the same > change. The Post-Execution Reflection at the bottom says what to check. **The whole idea in one line: for an arXiv paper, the PDF is a rendering of something you can get losslessly, so do not read the rendering.** `arxiv.org/e-print/<id>` serves the authors' own LaTeX. It carries the exact formulas, their `\label`s, and — critically — which formulas are _stated propositions_ rather than intermediate steps inside a proof. None of that survives into a PDF, so no vision model can recover it. Measured on one real paper, OCR of the PDF reached **0.958 mean token similarity** to the source and **only 49 % of its formulas compiled**, against 98 % for the source. Full numbers, including a systematic model defect that similarity cannot see, in [`../../references/OCR-VERSUS-AUTHOR-LATEX-GROUND-TRUTH.md`](../../references/OCR-VERSUS-AUTHOR-LATEX-GROUND-TRUTH.md). --- ## Step 1 — fetch the source, not the PDF Identify yourself; arXiv asks for it and rate-limits anonymous bulk access. ```bash ID=2605.00501 UA="your-project/1.0 (mailto:you@example.com)" mkdir -p /tmp/paper && cd /tmp/paper curl -sSL -A "$UA" "https://export.arxiv.org/api/query?id_list=$ID" -o meta.xml curl -sSL -A "$UA" "https://arxiv.org/e-print/$ID" -o eprint.tar.gz mkdir -p src && tar xzf eprint.tar.gz -C src ``` If `tar` fails the e-print may be a single gzipped `.tex` — `gunzip ...