fix-broken-scraperlisted
Install: claude install-skill prashant-cr/skills
# Fix a broken scraper
Scrapers break constantly, and almost all of the cost is misdiagnosis. "I'm being blocked" is the
loudest hypothesis, so people reach for headless browsers, proxies and spoofed headers — while the
page returns HTTP 200 and the selectors simply no longer match. Each wrong guess is permanent: a
browser added for a markup change stays in the stack forever, slower and more fragile than the code
it replaced, and it never addressed the fault.
So this skill classifies before it repairs, and changes one thing at a time.
## Workflow
### 1. Get the symptom precisely
Vague symptoms produce vague fixes. Pin down:
- **What exactly comes back?** Empty list, `None`, a partial row, wrong values, an exception, a
non-200 status — these are different failures with different causes.
- **All URLs or some?** A subset failing points at page variants, not at blocking.
- **Since when, and what changed?** A site redesign, a dependency upgrade, a new deployment, or a
changed IP each suggest a different class. "It worked yesterday" is worth pinning to a date.
- **First request or after N?** Failing immediately is a different problem from failing after two
hundred requests, which is pacing or session state.
Ask when these aren't given. The answers routinely make the diagnosis obvious before any request.
### 2. Reproduce at the smallest scale
Get to a single failing URL and a single extraction step. A scraper that fails inside a Scrapy
pipeline, a retry wrapper and a