site-extractlisted

# site-extract Scrapling-backed fetcher for pages WebFetch can't reach. **Fallback layer only** — WebFetch stays the default; reach for this when it comes back empty, blocked, or JS-gated. ## When to use 1. WebFetch returned a bot-check / access-denied / 403 / 503 / timeout. 2. WebFetch returned only static chrome (menus) but the real content is JS-rendered. 3. You need structured fields (CSS/XPath) rather than a markdown dump. ## When NOT to use - WebFetch already returned what you need → don't burn ~70s on a browser. - Mass crawling / many pages → out of scope (single-request tool, ToS risk). - Akamai/PerimeterX-class hard bot-walls (e.g. Mouser) → this often still fails; report the failure honestly, don't claim success. ## Run it The venv is self-contained at `./venv`. Always invoke via that interpreter: ```bash SK=~/.claude/skills/site-extract "$SK/venv/bin/python" "$SK/fetch.py" "<URL>" [options] ``` Options: - `--mode auto|static|stealth|dynamic|jina|firecrawl|reader` (default `auto` = static→stealth escalate) - `--jina-fast` — jina/firecrawl: skip JS-render wait (faster, static pages only) - `--css 'SELECTOR'` — repeatable; forces JSON output with per-selector arrays - `--format text|json|html` (default `text`) - `--timeout 70000` (ms), `--max-chars 20000` - `--ignore-robots` — only with explicit user authorization ### `jina` mode — Akamai/봇월 우회 (Jina Reader 프록시) `StealthyFetcher`도 못 뚫는 하드월(미스미 등)용. `https://r.jina.ai/<URL>`이 서버사이드 렌더 → 마크다운 반환. 기본 `X-Engine: