scraperapi-asynclisted

Product-usage reference for ScraperAPI's Async Jobs API — submit scraping jobs in the background and retrieve results via polling or webhook, including batch jobs up to 50,000 URLs. Consult when the user is scraping many URLs, needs non-blocking requests, or wants webhook delivery. Use when user asks: "how do I scrape 1000 URLs with ScraperAPI", "ScraperAPI async jobs", "batch scraping with ScraperAPI", "submit a scraping job and poll for results", "ScraperAPI webhook callback", "scrape URLs in the background", "ScraperAPI batchjobs endpoint". Covers single jobs, batch jobs (up to 50k URLs), webhook callbacks, all apiParams, async-exclusive parameters, binary response decoding, retention policy, and error handling.
scraperapi/scraperapi-skills · ★ 9 · API & Backend · score 78

Install: claude install-skill scraperapi/scraperapi-skills

# ScraperAPI Async Jobs API The Async API submits scraping jobs in the background and retries them for up to 24 hours to maximize success. Results are retrieved by polling a status URL or received automatically via webhook. ## When NOT to use Async - **Single URL, result needed immediately** → use the Standard API (`api.scraperapi.com`) — simpler and returns inline. - **Need to follow links across a site** → use the [Crawler](https://docs.scraperapi.com/crawler). - **Need recurring scheduled scraping** → use [DataPipeline](https://docs.scraperapi.com/data-pipeline). Use Async when: scraping 20+ URLs, the target site is slow or flaky, you want webhook delivery, or you need to scrape PDFs/images. ## Endpoints | Action | Method | URL | |--------|--------|-----| | Submit single job | POST | `https://async.scraperapi.com/jobs` | | Submit batch (up to 50k) | POST | `https://async.scraperapi.com/batchjobs` | | Check / retrieve job | GET | `https://async.scraperapi.com/jobs/<jobId>` | | Cancel job | DELETE | `https://async.scraperapi.com/jobs/<jobId>` | Auth: `apiKey` in the JSON request body (note: `apiKey` camelCase, unlike the Standard API's `api_key`). ## Single Job ```python import os, requests, time API_KEY = os.environ["SCRAPERAPI_API_KEY"] # Submit r = requests.post( "https://async.scraperapi.com/jobs", json={ "apiKey": API_KEY, "url": "https://example.com/product/123", "apiParams": { "render": True,