scraperapi-scraper-builderlisted

Build and implement web scrapers using ScraperAPI. Use this skill whenever the user asks to build, write, create, or implement a scraper, or wants runnable code that extracts data from a website. Trigger on: "build me a scraper for [website]", "write a scraper that fetches product pages from [ecommerce site]", "I need to scrape [data] from [website]", "create a script that extracts [fields] from [URL]", "help me scrape [website] — I need [fields]", "write code to scrape [website]", "make a script that scrapes [website]", "implement a scraper for [URL]". Guides architectural decisions (structured endpoint vs. raw HTML, JS rendering, proxy tier, sync vs. async batch), then generates a complete runnable Python or Node.js script with retry logic, error handling, pagination, and credit estimation.
scraperapi/scraperapi-skills · ★ 9 · Web & Frontend · score 78

Install: claude install-skill scraperapi/scraperapi-skills

# ScraperAPI Scraper Builder Build production-quality scrapers using ScraperAPI. Work through four phases: gather requirements, probe the target site, make architecture decisions, then generate a complete runnable script. ## Phase 1 — Gather Requirements Before writing code, collect the following. Pull from the user's prompt; ask only what is missing. | Info | Default if not specified | |------|--------------------------| | Target URL or website | Required — ask if missing | | Data fields to extract | Ask if vague ("product info" → which fields exactly?) | | Volume | Infer: single URL / paginated / bulk list of URLs | | Language | Ask if not clear from context; Python is a reasonable default | | Output format | stdout JSON | | Geo-targeting needed? | Infer from site type; confirm for e-commerce pricing | --- ## Phase 2 — Site Reconnaissance **Before making any architecture decisions, fetch 1–2 sample pages from the target site using ScraperAPI** to observe actual behavior. This replaces guesswork with evidence and costs at most 2 credits. ### What to fetch Fetch at most two pages: 1. **A listing or category page** (the main target) — e.g., a product grid, search results, or article index 2. **A detail page** (only if the user needs data from individual items) — e.g., a single product, article, or profile Always start with a **standard request** (no `render`, no `premium`) — the cheapest probe: ``` GET https://api.scraperapi.com/?api_key=<SCRAPERAPI_API_KEY>&url=<ta