authoring-airflow-dagslisted

# Authoring Airflow DAGs ## When to use - Creating or refactoring Airflow DAGs and tasks. - Configuring schedules, catchup/backfill, retries, and SLAs. - Passing data between tasks (XCom) or using connections/variables. - Do NOT use for diagnosing a broken running DAG (use `debugging-airflow-pipelines`). ## Workflow ``` - [ ] Make each task idempotent and parameterized by the data interval - [ ] Keep expensive/import-heavy code inside tasks, not at module top level - [ ] Set schedule + catchup deliberately - [ ] Configure retries, retry_delay, and SLAs - [ ] Wire dependencies via TaskFlow return values or >> operators ``` 1. **Idempotent tasks** — a task for the `2026-01-15` interval must produce the same result whether it runs once or is re-run. Use the data interval, not `datetime.now()`. 2. **No heavy top-level code** — the scheduler parses every DAG file frequently; database calls, API calls, or big imports at module level slow scheduling and can break parsing. Put them inside tasks. 3. **Schedule + catchup on purpose** — `catchup=True` backfills every missed interval from `start_date`; default to `False` unless you want that. 4. **Retries and SLAs** — transient failures are normal; set `retries` and `retry_delay`; use SLAs/alerts for lateness. ## Patterns **TaskFlow DAG, idempotent and cleanly wired:** ```python from airflow.decorators import dag, task import pendulum @dag( schedule="@daily", start_date=pendulum.datetime(2026, 1, 1, t