background-job-patternslisted

# Background Job & Queue Patterns ## Overview Every queue system — regardless of language or backend (Redis, Postgres, a message broker) — solves the same core problem: **deliver work at least once to a worker that might crash, hang, or run twice.** The same 14 patterns fix this in any stack; only the syntax changes. **Core principle:** the pattern is the reusable unit, not any one library's implementation. ## When to Use - Reviewing code that enqueues or processes jobs, in any language - Diagnosing: duplicate side effect on retry, job stuck "active"/"busy" forever, retries landing in a synchronized burst against an external API, worker memory that only climbs, a job silently missing after a deploy, DB connection pool exhausted by workers - Choosing a queue architecture (Redis vs Postgres `SKIP LOCKED` vs a real message broker) ## Quick Reference | Pattern | Prevents | Detail | |---|---|---| | Idempotency key | Duplicate side effects on retry | patterns.md #1 | | Visibility timeout / lease | Job "lost" if worker dies mid-run | patterns.md #2 | | Dead-letter queue | Infinite retry loop | patterns.md #3 | | Backoff + jitter | Retry storm / thundering herd | patterns.md #4 | | Concurrency ≤ weakest downstream resource | Connection exhaustion | patterns.md #5 | | Transactional outbox | Job fired before commit, or lost after a crash | patterns.md #7 | | Competing consumers | Throughput capped by a single worker | patterns.md #13 | | Circuit breaker | Retrying against a depe