ddia-systems

Solid

Design data systems by understanding storage engines, replication, partitioning, transactions, and consistency models. Use when the user mentions "database choice", "replication lag", "partitioning strategy", "consistency vs availability", "stream processing", "ACID transactions", "eventual consistency", or "LSM tree vs B-tree". Also trigger when choosing between SQL and NoSQL, designing data pipelines, or debugging distributed system consistency issues. Covers data models, batch/stream processing, and distributed consensus. For system design, see system-design. For resilience, see release-it.

AI & Automation 1,169 stars 127 forks Updated 2 weeks ago MIT

Install

View on GitHub

Quality Score: 94/100

Stars 20%
100
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Designing Data-Intensive Applications Framework A principled approach to building reliable, scalable, and maintainable data systems. Apply these principles when choosing databases, designing schemas, architecting distributed systems, or reasoning about consistency and fault tolerance. ## Core Principle **Data outlives code.** Applications are rewritten, languages change, frameworks come and go -- but data and its structure persist for decades. Every architectural decision must prioritize the long-term correctness, durability, and evolvability of the data layer above all else. **The foundation:** Most applications are data-intensive, not compute-intensive. The hard problems are the amount of data, its complexity, and the speed at which it changes. Understanding the trade-offs between consistency, availability, partition tolerance, latency, and throughput is what separates robust systems from fragile ones. ## Scoring **Goal: 10/10.** When reviewing or designing data architectures, rate them 0-10 based on adherence to the principles below. A 10/10 means deliberate trade-off choices for data models, storage engines, replication, partitioning, transactions, and processing pipelines; lower scores indicate accidental complexity or ignored failure modes. Always provide the current score and specific improvements needed to reach 10/10. ## The DDIA Framework Seven domains for reasoning about data-intensive systems: ### 1. Data Models and Query Languages **Core concept:** Th...

Details

Author
wondelai
Repository
wondelai/skills
Created
4 months ago
Last Updated
2 weeks ago
Language
Shell
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

ddia-principles

Designing Data-Intensive Applications (DDIA) distilled reference guide by Martin Kleppmann. MUST be loaded when: designing database schemas, choosing storage engines, implementing replication or partitioning, handling distributed transactions, building batch/stream processing pipelines, choosing consistency models, implementing consensus, designing data flow architectures, evaluating trade-offs between availability and consistency, encoding/serialization decisions, data modeling (relational vs document vs graph), building fault-tolerant systems, or any system design and architecture discussion involving data-intensive applications. Trigger on: database design, replication, partitioning, sharding, transactions, isolation levels, consistency, consensus, CAP theorem, batch processing, stream processing, MapReduce, Kafka, event sourcing, CDC, OLTP, OLAP, B-tree, LSM-tree, data warehouse, schema evolution, encoding formats, distributed systems, fault tolerance, leader election, quorum.

1 Updated today
satbirbhbc-ux
Web & Frontend Solid

system-design

Design scalable distributed systems using structured approaches for load balancing, caching, database scaling, and message queues. Use when the user mentions "system design", "scale this", "high availability", "rate limiter", "design a URL shortener", "system design interview", "capacity planning", or "distributed architecture". Also trigger when estimating infrastructure requirements, choosing between microservices and monoliths, or designing for millions of concurrent users. Covers common system designs and back-of-the-envelope estimation. For data fundamentals, see ddia-systems. For resilience, see release-it.

1,169 Updated 2 weeks ago
wondelai
Web & Frontend Solid

domain-driven-design

Model software around the business domain using bounded contexts, aggregates, and ubiquitous language. Use when the user mentions "domain modeling", "bounded context", "aggregate root", "ubiquitous language", "anti-corruption layer", "context mapping", "domain events", or "strategic design". Also trigger when splitting a monolith into services, defining microservice boundaries, or aligning code structure with business processes. Covers entities vs value objects, domain events, and context mapping strategies. For architecture layers, see clean-architecture. For complexity, see software-design-philosophy.

1,169 Updated 2 weeks ago
wondelai
AI & Automation Listed

principle-distributed-systems

Distributed systems principles — CAP, PACELC, consistency models (linearizable, causal, eventual, read-your-writes), consensus (Paxos, Raft), quorum, leader election, split-brain, replication, partitioning, gossip, logical clocks (Lamport, vector, hybrid), clock skew, delivery semantics (at-most-once, at-least-once, exactly-once effects), idempotency across nodes, two-generals problem, fallacies of distributed computing. Auto-load when reasoning about CAP/PACELC trade-offs, choosing a consistency model, designing consensus or leader election, sizing quorums, ordering events with logical clocks, distinguishing exactly-once delivery from exactly-once effects, designing replication or partitioning strategy, or assessing distributed failure modes.

2 Updated today
lugassawan
API & Backend Listed

database-design-patterns

Database schema design patterns and optimization strategies for relational and NoSQL databases. Use when designing database schemas, optimizing query performance, or implementing data persistence layers at scale.

15 Updated 2 days ago
NickCrew