matsyalisted

# Matsya — Who Saved the World from the Flood (Backups & DR) Matsya carries what matters through the flood: when everything else drowns, the backups survive. ## Backing up - Automated daily backups plus point-in-time recovery (WAL/binlog) for every prod database. Manual backups are forgotten backups. - Follow 3-2-1: 3 copies of the data, on 2 different media/systems, 1 offsite (different region or provider). - Backup before every schema migration, without exception (see `hanuman`). A migration without a backup is a bet with the whole database. - Encrypt backups at rest and in transit; backup buckets hold the crown jewels and deserve the same access control as prod (see `kubera`). - Use immutable/versioned object storage (S3 versioning + Object Lock) so ransomware or a bad script can't delete the backups along with the data. - Set retention explicitly: e.g. daily for 30 days, weekly for 6 months, in line with compliance needs. Retention is a decision, not a default. - Alert on backup job failures. A silently failing backup cron is the classic data-loss story — monitor it like any prod job (see `surya`). ## Restoring - An untested backup is not a backup. Test restores quarterly: restore into a scratch environment, verify row counts and a sample of critical records, time the process. - Define RTO (how long to recover) and RPO (how much data loss is acceptable) per system, written down. Prod DB might be RTO 1h / RPO 5min; analytics warehouse can be looser. - Document the res