Which of these do I need?
The Distributed Systems section of the pattern palette holds ten nodes, and a design that uses all ten usually means someone recognised the names before they had the problems. Each primitive answers one question that a system spread over several machines sooner or later has to ask. Start from the question and the primitive follows, and sometimes it turns out the question never comes up.
A question for each primitive
- Where does this key live? Consistent Hashing maps keys to nodes so that adding or removing a node moves only a small share of them; the ring is built step by step in Consistent hashing, and splitting data by range or by hash in general is Sharding & partitioning.
- How many copies, and which one takes the writes? Leader–Follower Replication: one node orders the writes and the others copy them, as in Replication & read replicas.
- How many copies must answer? Quorum: with N copies, a write waits for W of them and a read for R. Replication and quorum settings are where a system trades availability against consistency; Replication & read replicas also shows why R + W > N on its own is not strong consistency.
- How do several machines settle on one value, or one order of writes, while some of them fail? Consensus, a majority agreeing on each entry of a replicated log. It has no palette node because it usually lives inside another component, and it is not the same thing as a quorum read: Consensus.
- Which small facts must the whole cluster agree on? Who leads, who holds a lock, what the config says: a coordination service, built on consensus. What to keep in one is in Coordination services: small state, leases & watches.
- Who is in the cluster, and who is still alive? Gossip Protocol spreads membership and failure suspicion from peer to peer, with no central list (Gossip Protocol).
- Did two replicas change the same item without seeing each other? Vector Clock detects it; CRDT is a data type whose concurrent updates always merge. Conflict resolution for concurrent writes takes both.
- Will an acknowledged write survive this node crashing? The usual answer is a Write-Ahead Log: the write is appended and flushed before it is acknowledged and before the data files change (Write-Ahead Log).
- Could this key be in that file at all? A Bloom Filter says "definitely not" or "possibly" in little memory, which lets a lookup skip storage that cannot hold the key; it sits among the Data structures behind big systems.
- Must a change commit on every participant or on none? That is Two-Phase Commit, and its blocking coordinator is why Two-phase commit is worth reading before you use it.
- What if the old leader is still writing? Split-Brain Resolver: stop the side that should no longer act. Fencing tokens are in Distributed locks & leader election, and choosing the one instance that acts in Leader Election.
Agree first, or reconcile later
Most of these primitives line up behind one decision: does a write wait for agreement before it is acknowledged, or is it accepted and reconciled afterwards? Two designs of the same replicated key-value store show how that choice drags the rest of the map along with it.
The leaderless design, from Amazon's Dynamo paper of 2007, reconciles later. Consistent hashing places each key, a quorum decides when a write counts, gossip tracks who is alive, and vector clocks catch the conflicting versions this design lets in. It keeps taking writes through node failures, as long as W nodes answer, and pays in conflicts the application may have to settle.
Each node is a technique, not a service: the edge labels say what it contributes to this write.
The leader-based design agrees first: each range of keys is a replicated log run by consensus, as in CockroachDB's Raft group per range. One order per range makes vector clocks and CRDTs unnecessary, and brings in leases and fencing to silence a deposed leader. It never returns conflicting versions, and pays with a minority side of a partition that cannot write.
Which design fits turns on whether, when nodes fail or the network splits, a stale or conflicting answer is worse than no answer at all, the trade-off laid out in Consistency models: strong vs eventual (CAP).
Composing them under a brief
No node on the Distributed Key-Value Store palette carries a primitive's name, which makes it a fair test of this map. The brief asks for the leaderless design: consistent hashing, replication, quorum reads and writes, conflict detection with vector clocks and read repair, and nodes that join, leave and fail without downtime. Every one of those has to surface as a decision you can point to, written on the drawing: N, W and R beside the replication, the version that travels with each value, what a returning node does to catch up. Consistent Hashing, Vector Clocks and CRDT Replication each isolate a single primitive, if one of them is the part you want to practise first.