← ClaudeAtlas

backuplisted

Back up expensive-to-regenerate pipeline/project artifacts to Dropbox with the right format and location. Use when the user wants to offload, back up, or archive data to Dropbox — a corpus, a cache, scraped raw, or a whole pipeline's build outputs.
hsigstad/research-kit · ★ 0 · Data & Documents · score 70
Install: claude install-skill hsigstad/research-kit
# Back up data to Dropbox Disaster-recovery copies of expensive-to-regenerate artifacts (scraped corpora, LLM caches, geoblocked raw PDFs, BigQuery exports). Dropbox is a **cold backup**, not a working store. For ad-hoc file sharing / browsing, use the `dropbox` skill instead. Writable remotes exist only on the **host** (`bi-dropbox:`), not the sandbox (`*-ro:` only). Stage compressed archives on `/projects` (has TBs free), not `/tmp`. ## The two decisions: FORMAT and LOCATION ### FORMAT — keyed on file count/size, not one-size-fits-all The dividing line is Dropbox's `too_many_write_operations` failure (bursts of many small files). Pick by artifact shape: | Shape | Form | Why | |---|---|---| | **Corpus**: hundreds–low-thousands of medium/large files (parquet, MB–GB each) | **Per-file**, recompressed to **ZSTD-9**, mirroring the local tree | Directly readable + partial restore; `rclone copy` gives free **incremental sync**; ZSTD-in-parquet beats tar-of-snappy (a real case: 44 GB SNAPPY → 18 GB ZSTD vs 34 GB tar.zst) | | **Tiny-file cache**: 10k–100k files, few KB each (LLM caches) | **Single `tar.zst`** | Per-file trips the write-op limit; FS block-slack inflates size (a "1 GB" cache was 411 MB of data across 63k files → 20 MB zstd) | | **Bulk incompressible**: 10k+ PDFs/images | **`tar` with `zstd -3`** (combine, don't compress) | Compression won't help; combining dodges the write-op limit | Rule of thumb: per-file is strictly better **until** files get too many/too ti