scraperapi-research-agentlisted
Install: claude install-skill scraperapi/scraperapi-skills
# ScraperAPI Research Agent
End-to-end autonomous research: ScraperAPI finds and fetches sources → Anthropic Files API
ingests them as cited documents → Claude synthesizes a report.
**Run it:**
```bash
# Install dependencies
pip install requests anthropic
# Set env vars
export SCRAPERAPI_API_KEY=your-key
export ANTHROPIC_API_KEY=your-key
# Run
python skills/scraperapi-research-agent/scripts/research_agent.py \
--question "What are the best practices for rate limiting in web APIs?" \
--max-sources 5 \
--output report.md
```
See [scripts/research_agent.py](scripts/research_agent.py) for the full implementation.
---
## Planning Checklist
Before starting a research run, establish:
- [ ] **Question clarity** — Is the question specific enough to produce useful search queries? Vague questions like "tell me about AI" produce noise. Better: "What are the tradeoffs between RAG and fine-tuning for domain-specific LLMs?"
- [ ] **Source count** — How many sources are needed? 3–5 is usually sufficient for a factual summary; 8–10 for a comparative analysis. More sources = more ScraperAPI credits.
- [ ] **Recency** — Does the answer depend on recent events? Search queries will use recent date filters.
- [ ] **Credit budget** — Each source costs ~1 credit to scrape (more with JS rendering). 5 sources = ~5–10 credits total.
- [ ] **Stop condition** — Define when to stop. The default stop is `--max-sources` (5). Do not loop indefinitely.
---
## Research Loop
```
1. PLAN
↓