scrape-batchlisted
Install: claude install-skill n24q02m/claude-plugins
# scrape-batch
Fan out `extract(action="batch")` over a URL list wet already knows, then
report per-URL outcomes honestly. The batch path applies per-domain
politeness (2 concurrent and 1 request/second per domain, 6 fetches in
flight overall) and returns whatever succeeded even when some URLs fail.
Use this skill when:
- The user supplies a list of URLs to read in full.
- A previous `search` returned hits and the user wants the bodies, not
the snippets.
- A crawl or `extract(action="map")` produced a URL set to pull down.
Do NOT use this skill when:
- There is one URL, or a handful from one domain -- call
`extract(action="extract", urls=[...])`, which is cached and cheaper.
- The URLs are not known yet and the goal is an answer, not the pages --
use the `research-topic` skill (`extract(action="agent")`).
- The target is a whole site rather than a list -- use
`extract(action="crawl")` or `extract(action="map")`.
- The user wants images or video from the pages -- use
`media(action="list")` then `media(action="download")`.
## Steps
1. **Collect and de-duplicate the URL list.** Drop duplicates and
fragment-only variants (`#section`) -- each one costs a full fetch.
Report the final count to the user before spending it.
2. **Split into chunks of at most 50.** The cap is hard: 51 URLs returns
`{"error": "Error: Maximum 50 URLs per batch (got 51)"}` and nothing
is fetched -- the call is refused, not truncated.
3. **Split further when one domain dominates