ai-web-scraping-scrapegraphlisted

# ScrapeGraph AI - Intelligent Web Scraping ## Setup Read your credentials from ~/.gooseworks/credentials.json: ```bash export GOOSEWORKS_API_KEY=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json'))['api_key'])") export GOOSEWORKS_API_BASE=$(python3 -c "import json;print(json.load(open('$HOME/.gooseworks/credentials.json')).get('api_base','https://api.gooseworks.ai'))") ``` If ~/.gooseworks/credentials.json does not exist, tell the user to run: `npx gooseworks login` All endpoints use Bearer auth: `-H "Authorization: Bearer $GOOSEWORKS_API_KEY"` Extract web content using AI with natural language prompts. ## Capabilities - **Start SmartScraper**: Extract content from a webpage using AI by providing a natural language prompt and a URL - **Start SearchScraper**: Start a new AI-powered web search request - **Scrape**: Extract raw HTML content from web pages with JavaScript rendering support - **Start SmartCrawler**: Start a new web crawl request with AI extraction or markdown conversion - **Start Sitemap**: Extract all URLs from a website sitemap automatically - **Start Markdownify**: Convert any webpage into clean, readable Markdown format - **Get SearchScraper Status**: Get the status and results of a previous search request (free) - **Get Markdownify Status**: Check the status and retrieve results of a Markdownify request (free) - **Get Sitemap Status**: Check the status and retrieve results of a Sitemap request (free) - **Get SmartCraw