Replication Topologies: Leader–Follower, Multi-Leader, and Leaderless Systems

Why Replication Exists—and What Every Replica Costs You

A single data node forces three painful choices. If it dies, the data or the service may be unavailable. If every read and write goes through it, traffic concentrates there instead of scaling across the cluster. If users are far from it, every request pays the distance to one location. Replication addresses these pressures by keeping copies of the same data on multiple nodes: a surviving copy supports recovery, multiple copies can serve reads, and a nearby copy can reduce network distance. HDFS describes replica placement as a way to improve “data reliability, availability, and network bandwidth utilization”; it also notes that placing replicas across racks increases write cost because each write must transfer data to multiple racks.

The cost is not just storage or network traffic. Once the same record exists on several nodes, those nodes can disagree temporarily. You must decide which copy a reader may see, how soon an update becomes visible, and what happens when a copy is unreachable. As the leader-and-follower pattern puts it, when data is updated on multiple servers, “you need to decide when to make it visible to clients.” Replication therefore exchanges a single-node failure boundary for a distributed consistency decision: more copies improve resilience and reach, but they create more versions whose ordering and freshness must be managed.

Consider a ride-sharing service. Driver-location writes pour in from vehicles across the world, while riders and dispatchers read those locations regionally. With one leader in one datacenter, every location update funnels through that site. The leader becomes a write bottleneck, and its failure becomes a single point of failure for location updates. Replicas can spread regional reads and keep service running through a node failure, but a regional reader may still hold an older location until the update reaches it.

That workload exposes the design choices ahead. A single-leader topology keeps the write order and conflict surface comparatively simple, but it trades off availability when the leader or its path fails. Multi-leader replication places writable leaders in multiple datacenters, improving geographic write access while opening the conflict problem: independent leaders can accept incompatible updates. Leaderless replication removes the special write owner and makes the client participate in consistency through read and write choices. The flexibility increases, but so does the failure-handling responsibility at the edges of the system.

Replication's job is to place enough copies of data across nodes to survive node loss, spread reads, and bring data closer to users—while making freshness and disagreement explicit design decisions.

Leader–Follower Replication: One Write Path, Many Read Copies

Leader–follower replication puts one node in charge of ordering writes. The leader accepts the client update, records it in a replication log, and ships that record to followers. Followers apply the record to their local copies and can serve reads; they also provide candidates for failover if the leader becomes unavailable. The key invariant is simple: there is one write path, so the system does not need to reconcile two leaders making independent decisions.

The write path

A write first reaches the leader, not an arbitrary follower. The leader appends the update to its replication log before deciding when to acknowledge the client. That log may be a storage engine's write-ahead log or a logical or row-based change log. The log record is the unit shipped to followers: it contains enough information for each follower to apply the same change in the same order.

The acknowledgement point is the principal design choice. In synchronous mode, the leader waits for the required follower acknowledgments before acknowledging the write. This couples the client's write latency and availability to the followers that must acknowledge: if one stalls, the leader can no longer complete the write under the stated policy. The benefit is a stronger durability boundary because the update has reached the required replicas before the client is told it committed.

Write the update.
Append the update to the leader's replication log.
Send the log record to followers.

If replication is synchronous:
    Wait for the required follower acknowledgments.
    Acknowledge the write.

If replication is asynchronous:
    Acknowledge without waiting.
    Followers apply the record asynchronously.

The pseudocode makes the ordering explicit: append first, send second, then branch on mode. The asynchronous branch returns immediately after the leader records and sends the record; followers apply it later. That path keeps the leader from waiting for follower progress, but a leader crash can leave a committed client write only on the leader. The replacement leader may therefore be missing that record, which is replication lag expressed as possible loss of a committed write.

The read path and its consequence

A read can be routed to a follower to spread read load, but the answer is only as current as that follower's applied log position. A follower that has received a record but not applied it, or has not received it yet, returns an older version. The leader remains the natural route when the caller needs the write path's current state; routing reads to followers is a consistency decision, not merely a load-balancing decision.

Consider a comment service. A user posts a comment through the leader and immediately refreshes a feed served by a follower. If asynchronous shipping has not caught up, the refresh returns the old feed and the comment appears to vanish. A practical read-your-own-writes policy routes that user's reads to the leader for a short window after the write, or otherwise keeps the user's reads on a replica known to have passed the write. The exact window and routing mechanism are application decisions; the topology alone does not make a follower current.

Multi-Leader Replication: Independent Writes Across Datacenters

Multi-leader replication gives each datacenter a local write path. A client writes to the leader in its nearest or currently connected datacenter, so the request does not have to cross a wide-area network before it is accepted. The leaders then propagate their updates to one another asynchronously. This topology fits multiple datacenters, active-active high availability, and clients that must continue writing while offline, such as mobile applications or collaborative-document clients.

The trade-off is that there is no single ordered write stream. Two leaders can accept updates to the same record before either has received the other update. The system must therefore carry enough version information to determine whether one update happened before another or whether the versions are concurrent. A conflict is not a routing problem: a load balancer can choose a datacenter, but it does not merge the state stored in two datacenters.

The conflict-handling path

When a remote update arrives, compare its version metadata with the local version. If the metadata establishes that one update supersedes the other, retain the newer version; otherwise treat the updates as concurrent and apply the configured conflict policy. That decision belongs in the replication path, not in an implicit arrival-order rule: network delay can make the update that arrives first the one that was written second.

The following handler makes that branch explicit. It gives document titles a special application merge path, uses timestamp comparison for last-write-wins (LWW) where that policy is acceptable, and otherwise delegates to a merge function or a CRDT merge. The title branch matters because treating two concurrent edits as an ordered sequence can silently erase one user's valid change.

When a remote update arrives:
    Compare its version metadata with the local version.
    If one update supersedes the other, retain the newer version.
    Otherwise, treat the updates as concurrent.

For concurrent document-title edits:
    Apply an application-level merge, CRDT, or OT policy.
    Do not rely on arrival order.

For other concurrent updates:
    Apply the configured LWW, CRDT, or application-merge policy.
    With LWW, follow the timestamp policy and account for clock drift.

Four ways to resolve concurrent writes

• Last-write-wins (LWW): choose the value with the greatest timestamp. It is simple and deterministic for a given timestamp order, but it can discard a valid write. “Last” is usually tied to wall-clock time, and clock drift between machines can make the timestamp order disagree with the actual write order.
• Application-level merge: interpret both values using domain rules. A document service might preserve both title edits, ask the user to choose, or merge fields according to product semantics. The merge function is specific to the application.
• CRDTs: store the data in a structure whose merge operation is designed to combine concurrent states. The data type carries the conflict semantics, allowing replicas to converge without choosing one entire version as the winner.
• Operational transform (OT): transform concurrent editing operations against one another so both edits can be applied consistently. Collaborative editors use this kind of operation-aware approach rather than treating a document as an opaque last-write-wins value.

Leaderless Replication: Quorum Reads, Writes, and Repair

Leaderless replication removes the distinguished write owner: any replica can accept reads and writes. A request is coordinated by whichever replica receives it, and the coordinator contacts the other replicas that hold the key. This avoids depending on one node for request handling and can serve reads locally, but it shifts consistency work into the request path and the repair machinery.

The N, W, and R contract

For each key, N is the number of replicas in its preferred replica set. A write succeeds after at least W replicas acknowledge it; a read completes after at least R replicas respond. With W + R > N, the write set and read set must share at least one preferred replica. The overlap ensures that at least one read respondent participated in the write; version metadata is then used to compare responses and select a value according to the system's conflict-resolution policy.

A common balanced configuration is N=3, W=2, and R=2: two replicas must store a write before success, and two must answer a read. You can skew the thresholds toward R when reads dominate, or toward W when writes dominate. Lowering either threshold improves the corresponding operation's availability and latency, but reduces overlap. Even the intersection rule does not provide linearizability: concurrent writes can produce conflicting versions, and a read can race with propagation unless the system adds stronger coordination.

N = number of replicas in the preferred replica set
W = write acknowledgments required for success
R = read responses required for completion

Write:
    Send the update to the preferred replicas.
    If W acknowledgments arrive, finish the write.
    Otherwise, use a reachable substitute during a sloppy quorum.
    Retain a hint identifying the preferred owner for hinted handoff.

Read:
    Read from replicas until R responses are available.
    Compare returned versions using the system's version metadata.
    Select or merge a value according to the conflict policy.
    Send the selected value to stale replicas as read repair.

When a preferred owner recovers:
    Forward the hinted data to that owner.

The write path first targets the key's preferred replicas and returns success once W acknowledgements arrive; it does not need to wait for every replica. The read path collects R responses, selects the highest version, and returns that value. The version is essential: timestamps are vulnerable to clock skew, while logical clocks, vector clocks, or version vectors can provide ordering or conflict information. The important boundary is that quorum arithmetic tells you where sets intersect; version tracking tells you what to do when the responses disagree.

Repair closes the gaps

Quorum completion is not the same as convergence. Read repair fixes divergence opportunistically: when a read discovers that one respondent has an older version, the coordinator sends the newest value back to that stale replica. This repairs keys that are read, but an untouched key can remain divergent. Anti-entropy handles that blind spot in the background by comparing replicas and synchronizing differences. A Merkle tree makes the comparison selective: matching root hashes mean the compared data is identical; differing hashes guide the replicas toward the divergent keys instead of transferring the full dataset.

During a partition, a strict quorum can fail because a preferred replica is unreachable. A sloppy quorum temporarily sends the write to a reachable node that is not one of the key's preferred replicas. That substitute stores a hint naming the rightful owner. When the owner recovers, hinted handoff forwards the data and deletes the hint. This favors availability, but the original W + R > N proof no longer guarantees that a read intersects the nodes that accepted the write; anti-entropy remains necessary for complete convergence.

Worked Example: A Dynamo-Style Shopping Cart

A Dynamo-style shopping cart has no single leader that owns the record. Any replica can accept a client write, so two clients can update the same cart concurrently at different replicas. Amazon Dynamo popularized this combination of leaderless replication, tunable N, W, and R, vector clocks, read repair, and anti-entropy.

Use three replicas for the cart: N = 3. Client A writes item A to one replica while Client B writes item B to another. Neither write overwrites the other immediately because the replicas have observed different versions. A later read collects both versions, {A} and {B}, and the application merges them deterministically with set union: {A} ∪ {B} = {A, B}. The resulting cart is a CRDT for additions: the merge operation preserves both independently added items and converges when replicas exchange the merged state.

Check the quorum arithmetic

Configure each cart write to wait for W = 2 acknowledgements and each read to collect R = 2 responses. The intersection calculation is W + R = 2 + 2 = 4; because 4 > N = 3, every two-node read set must overlap the two-node write set in at least one replica. That overlap gives the read a copy that participated in the successful write, which can be identified as the newest version using version metadata.

The comparison matters. With W = 1 and R = 1, W + R = 1 + 1 = 2, and 2 is not greater than N = 3. A successful write may reach only one replica, while a later one-node read reaches either of the other two; that read can miss the write entirely. The lower thresholds improve availability and reduce coordination, but they remove the intersection guarantee.

Why deletion needs history

Suppose the merged cart is {A, B} and a client removes A. Physical removal is unsafe: a replica that still holds {A, B} could later exchange its older state and reintroduce A. Instead, the delete writes a tombstone for A, such as “A deleted,” and replication carries that marker until every relevant replica has learned the deletion. The tombstone lets the merge distinguish “A was intentionally removed” from “this replica has never seen A.” The tombstone must remain long enough to prevent older copies from reintroducing the deleted item; its removal requires a safe deletion-retention policy.

Choosing a Replication Topology

Choose the topology from the consistency requirement outward. If a banking balance or inventory count needs linearizable reads, prefer a single leader with synchronous replication, or add a consensus layer such as Raft or Paxos. You are accepting write coordination and reduced availability during some failures in exchange for one ordered view of the data. In CAP terms, a partition forces the design to choose between continuing to serve operations and preserving that single consistent view. If users must write with low latency from multiple geographic regions, a single leader makes those writes pay the round trip to that leader; multi-leader or leaderless replication favors continued local availability, but moves conflict detection and resolution into the system and often the application.

The comparison is not simply “centralized versus distributed.” It is about where the design puts ownership, coordination, and failure handling. Leader-follower concentrates writes and makes ordinary replication familiar, but failover becomes a critical transition. Multi-leader preserves local write availability across datacenters, while asynchronous propagation means concurrent updates need an explicit merge policy. Leaderless replication removes dependence on one leader and can keep serving through node failures, but clients and replicas must reason about quorum results, divergent versions, read repair, anti-entropy, sloppy quorums, and hinted handoff.

Read the price column before the capability column. Leader-follower buys an ordered write stream by limiting write ownership; its cost is geographic write latency, a leader write bottleneck, and failover complexity. Multi-leader buys low-latency local writes and availability across datacenters; its cost is application-specific conflict resolution. Leaderless buys flexible request routing and no leader election; its cost is coordination and client-side failure handling, with eventual convergence still weaker than linearizable reads. The topology that appears most available may therefore be the one that makes the product responsible for the hardest correctness decisions.

The article presents Postgres streaming replication and MySQL binlog replication as operationally well-understood examples around replication state and failover. Leaderless systems such as Cassandra and Dynamo avoid a single write owner, but the team must understand quorum math and failure recovery deeply enough to choose behavior for partitions and membership changes. A useful design review ends with three explicit answers: which reads must be linearizable, which failures may stop writes, and where conflicting writes are merged.

Where Replication Fails: Lag, Conflicts, and Failover

Asynchronous replication usually fails in the gap between the leader accepting a write and a follower applying it. The leader commits the change, ships its replication log, and continues serving traffic; a follower that is delayed by a network hiccup can still answer reads from its older state. This is a structural property of asynchronous replication, not merely a symptom of high load. Even a lightly loaded cluster can accumulate seconds of lag during a network interruption, long enough for any user's next read to observe stale data.

Lag becomes a user-visible anomaly

The simplest failure is a read-your-own-writes violation. A user posts a comment through the leader, then immediately refreshes their feed. If the refresh is routed to a lagging follower, that follower does not contain the comment, so the comment appears to vanish. Route that user's reads to the leader for a short window after the write, or keep them on a replica known to have caught up, if the product requires the read-your-own-writes guarantee.

Lag can also violate monotonic reads. A user can first read from a follower with a newer version, then reach another follower that is still behind and observe the data move backward. A causality violation is subtler: one request observes an event, then a dependent request reaches a replica that has not observed the event's cause. For example, the UI confirms the comment was created, while a subsequent feed or notification read behaves as if the creation never happened. These are different routing symptoms of the same missing guarantee: replicas do not become current at the same time.

Failover turns stale state into conflicting authority

Steady-state lag is usually visible as stale reads; failover can turn it into lost or conflicting writes. Suppose the leader becomes unreachable while a follower is behind. An operator or automated controller promotes the follower, but the new leader does not contain every committed change from the old one. If the old leader later recovers and continues accepting writes, two nodes now believe they are authoritative: this is split-brain. Clients can observe different values, and the two histories can diverge even after connectivity returns.

The dangerous sequence is therefore: detect the old leader as unavailable, choose a replacement from potentially stale followers, promote it, and prevent the old leader from serving writes. Promotion without fencing is unsafe because reachability is not the same as authority. The old leader may be isolated from the control plane but still reachable by some clients. Fencing must make that old identity reject writes before the replacement becomes active. The replacement decision must also account for follower freshness; otherwise failover converts replication lag into data loss.

This is why a cluster can look healthy for long periods and still fail during recovery. Normal replication has one write authority and a predictable propagation path. Failover changes the authority while messages may be delayed, nodes may have different histories, and clients may still hold routes to the old leader. The outage is concentrated in that transition rather than in ordinary steady state.

Failure Defenses: Fencing Leaders and Handling Conflicts

Split-brain: failover creates two authorities

Failover is unsafe when reachability is mistaken for authority. A leader becomes unreachable from part of the cluster. Different failover actors can promote different followers when they observe inconsistent views of the network. The old leader has not necessarily stopped: it may still accept writes from clients that can reach it. You now have multiple promoted nodes and the old node producing competing histories. Operators see divergent write streams, clients reading different values, and replication unable to converge without an explicit decision about which history survives.

Treat promotion as a two-part operation, not a role change. First, choose the candidate with the most complete replicated history; promoting a stale follower can discard committed writes that never reached it. Second, fence every other candidate, including the old leader, before the new leader serves writes. Use a fencing mechanism that prevents the old leader from accepting writes before the replacement serves writes. If the old leader later reconnects, it can copy state as a follower, but it cannot re-enter as an authority. The cost is an unavailable or read-only interval while the authority is established; that interruption is safer than accepting two histories.

Clock-based conflict resolution can erase valid writes

Multi-leader systems have a different failure boundary: two leaders can accept valid writes concurrently. Last-write-wins appears deterministic, but “last” is commonly derived from wall-clock time. Clocks on different machines can drift, so the write that happened later can carry an earlier timestamp and silently lose. A document title can therefore be changed to Report Q1 even when that value came from the later user action if clock skew makes its timestamp appear older. Do not use LWW for fields where losing one valid update is unacceptable; choose an application merge, or use a data structure whose merge semantics are designed for concurrent updates. The defense belongs with the field's meaning, not merely with the replication transport.

Defending the Choice in a Design Interview

A strong design answer starts with the consistency requirement, not the replication topology. Ask: which operations must be linearizable, and which anomalies can the product tolerate? A banking balance and an inventory count usually need linearizable reads, pointing you toward a single leader with synchronous replication or toward a consensus layer such as Raft or Paxos. If the requirement is geo-distributed, low-latency writes, say explicitly that you are choosing multi-leader or leaderless replication—and that the design now owns conflict handling.

For a ride-sharing service, driver-location writes arrive in multiple regions and reads are regional. A single leader may simplify ordering, but it concentrates write traffic and makes cross-region write latency part of the path. Multiple leaders reduce the distance to a write, but concurrent updates can conflict. A leaderless design removes the single write authority, but clients and the storage layer must reason about quorum acknowledgements, stale reads, repair, and conflicting versions. The topology is not the requirement; it is the consequence of the requirement.

The question that separates operation from vocabulary is: “Walk me through the user-visible failure when one replica is behind, and tell me where the next read goes.” A useful answer names read-your-own-writes, monotonic-read, and causality anomalies rather than saying only “eventual consistency.” For example, after a profile update, a refresh can hit a stale follower and show the old value. You can mitigate that particular case by routing the user's reads to the leader for a short window after the write, but you still need a policy for other stale or conflicting reads.

Close by stating the operational price of the choice. Single-leader replication is generally operationally well-understood, but failover, stale followers, and write availability need explicit defenses. Multi-leader and leaderless designs buy write locality or availability by moving more failure handling into conflict resolution, clients, and quorum math. A weak answer says “eventual consistency is fine” or “we'll use active-active.” A strong answer names the tolerated anomalies, the mitigation for each, and the failure mode that would make the chosen guarantee unacceptable.