data-sourceslisted
Install: claude install-skill adamw7/tools
# Data Sources Skill
Pick the right data source in the `data` module, respect the schema contract
that keeps forward-only sources away from schema-dependent callers, and run the
uniqueness checker to find whether a set of columns can serve as a key.
## When to Use
- Reading tabular data from CSV, a JDBC query, Parquet, JSON, YAML, or TOON
- Adding a brand-new data source
- Checking whether columns are unique / finding a smaller key
- The user says "data source" / "uniqueness check" / "find a key"
## In-memory vs iterative — pick first
Every format ships in two variants:
- **`InMemory…`** — loads all rows once (`readAll()`), then runs multiple
recursive checks cheaply. Use when the data fits in heap.
- **`Iterable…`** — holds one row at a time (tiny heap), but re-reads the source
for each recursive pass. Use for large data or streaming.
## The schema contract (don't fight it)
- **`IterableDataSource`** — the base forward-only contract: `open()`,
`nextRow()`, `hasMoreData()`, `reset()`, `nextRows(int batchSize)`.
- **`ColumnarDataSource extends IterableDataSource`** — adds `getColumnNames()`
for sources whose columns are known up front.
- Forward-only sources that discover keys as they stream — **iterable JSON, YAML,
TOON** — deliberately do **not** implement `ColumnarDataSource`. Callers that
need the schema (e.g. the uniqueness check) depend on the narrower
`ColumnarDataSource`, so a schema-less source can never be handed in and answer
with `null`. Don't w