apify-core-workflow-b

# Apify Core Workflow B — Storage & Pipelines ## Overview Manage Apify's three storage types (datasets, key-value stores, request queues) and orchestrate multi-Actor pipelines. Covers CRUD operations, data export, pagination, and chaining Actors together. ## Prerequisites - `apify-client` installed and authenticated - Familiarity with `apify-core-workflow-a` ## Storage Types at a Glance | Storage | Best For | Analogy | Retention | |---------|----------|---------|-----------| | Dataset | Lists of similar items (products, pages) | Append-only table | 7 days (unnamed) | | Key-Value Store | Config, screenshots, summaries, any file | S3 bucket | 7 days (unnamed) | | Request Queue | URLs to crawl (managed by Crawlee) | Job queue | 7 days (unnamed) | Named storages persist indefinitely. Unnamed (default run) storages expire after 7 days. ## Instructions ### Step 1: Dataset Operations ```typescript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); // Create a named dataset (persists indefinitely) const dataset = await client.datasets().getOrCreate('product-catalog'); const dsClient = client.dataset(dataset.id); // Push items (single or batch) await dsClient.pushItems([ { sku: 'ABC123', name: 'Widget', price: 9.99 }, { sku: 'DEF456', name: 'Gadget', price: 19.99 }, ]); // List items with pagination const page1 = await dsClient.listItems({ limit: 100, offset: 0 }); console.log(`Total items: ${page1.total}, ...