Eventual Consistency

TL;DR

Eventual consistency is a consistency model where, in the absence of new updates, all replicas will eventually converge to the same value. It favors availability and partition tolerance over strong consistency (CAP theorem tradeoff), powering systems like DynamoDB, Cassandra, and DNS at massive scale.

Visual Overview

STRONG CONSISTENCY (All replicas synchronized immediately)
┌─────────────────────────────────────────────────────────┐
│  Client writes value = 42                               │
│         ↓                                               │
│    ┌─────────┐   sync    ┌─────────┐   sync   ┌──────┐│
│    │ Primary │ ========> │ Replica │ =======> │Repli-││
│    │ value=42│           │ value=42│          │ca=42 ││
│    └─────────┘           └─────────┘          └──────┘│
│         ↓                                               │
│    All replicas updated                                 │
│    THEN acknowledge write ✓                             │
│                                                         │
│  Trade-off: High latency, but reads always consistent   │
└─────────────────────────────────────────────────────────┘

EVENTUAL CONSISTENCY (Replicas sync asynchronously)
┌─────────────────────────────────────────────────────────┐
│  Client writes value = 42                               │
│         ↓                                               │
│    ┌─────────┐   async   ┌─────────┐   async  ┌──────┐│
│    │ Primary │ --------> │ Replica │ -------> │Repli-││
│    │ value=42│  (later)  │ value=?? │ (later) │ca=?? ││
│    └─────────┘           └─────────┘          └──────┘│
│         ↓                                               │
│    Acknowledge write immediately ✓                      │
│    (replicas update in background)                      │
│                                                         │
│  Eventually:                                            │
│    All replicas converge to value=42                    │
│                                                         │
│  Trade-off: Low latency, but temporary inconsistency    │
└─────────────────────────────────────────────────────────┘

TIMELINE VIEW:
T0: Write arrives at primary
T1: Primary acknowledges write ✓ (user sees success)
T2: Replica 1 receives update (100ms later)
T3: Replica 2 receives update (200ms later)
T4: All replicas converged (eventual consistency achieved)

Problem Window (T1-T4):
- Reads from replicas may return stale data
- Different replicas may return different values

Core Explanation

What is Eventual Consistency?

Eventual consistency is a consistency model used in distributed databases where updates propagate to all replicas asynchronously. The system guarantees that:

1. Eventually: Given enough time without new updates
2. All replicas converge: To the same final value
3. No guarantee on timing: Could take milliseconds or seconds

This is in contrast to strong consistency, where all replicas are updated synchronously before acknowledging a write.

Real-World Analogy:

Think of eventual consistency like Wikipedia edits:

• Someone updates an article (write to primary)
• The change is visible immediately on their screen
• Other users see the change after their browser cache expires (eventual propagation)
• Eventually, everyone sees the same updated article

The CAP Theorem Trade-off

Eventual consistency is a choice made in the CAP theorem:

CAP Theorem: Pick 2 of 3
┌─────────────────┬──────────────────┬──────────────────┐
│   Consistency   │  Availability    │   Partition      │
│                 │                  │   Tolerance      │
└─────────────────┴──────────────────┴──────────────────┘

Strong Consistency (CP):
├─ Consistency: ✓ All reads return latest write
├─ Availability: ✗ May block during network partition
└─ Partition Tolerance: ✓ Handles network failures

Eventual Consistency (AP):
├─ Consistency: ✗ Reads may return stale data
├─ Availability: ✓ Always accepts reads/writes
└─ Partition Tolerance: ✓ Handles network failures

When Network Partition Happens:

Scenario: Network partition splits system into two halves

STRONG CONSISTENCY (CP System):
┌──────────────────────────────────────────────────┐
│  Partition!                                      │
│   Group A         ║    Group B                   │
│  ┌────────┐      ║   ┌────────┐                │
│  │Primary │      ║   │Replica │                 │
│  └────────┘      ║   └────────┘                │
│      ↓           ║       ↓                       │
│  Accepts writes  ║   REJECTS writes             │
│  (has majority)  ║   (no majority, unavailable) │
└──────────────────────────────────────────────────┘
Result: System unavailable in Group B

EVENTUAL CONSISTENCY (AP System):
┌──────────────────────────────────────────────────┐
│  Partition!                                      │
│   Group A         ║    Group B                   │
│  ┌────────┐      ║   ┌────────┐                │
│  │Primary │      ║   │Replica │                 │
│  └────────┘      ║   └────────┘                │
│      ↓           ║       ↓                       │
│  Accepts writes  ║   Accepts writes             │
│  value=42        ║   value=99                    │
└──────────────────────────────────────────────────┘
Result: System available, but conflicting values!

After Partition Heals:
- Need conflict resolution (last-write-wins, vector clocks, etc.)

Consistency Levels

Most eventually consistent systems offer tunable consistency:

1. Read-Your-Writes Consistency

Guarantee: A user always sees their own writes

Example: Social media post
T0: Alice posts "Hello World"
T1: Alice refreshes page → sees "Hello World" ✓
T2: Bob (different server) may not see it yet ✗

Implementation:
- Route user's reads to same replica they wrote to
- Or use session tokens to track latest write timestamp

2. Monotonic Reads

Guarantee: Time never goes backward for a single user

Example: Message timestamps
T0: User sees message at timestamp 100
T1: User refreshes → cannot see older timestamp 50 ✗
T2: User refreshes → may see timestamp 100 or 150 ✓

Implementation:
- Track last-seen timestamp per user
- Only show updates after that timestamp

3. Monotonic Writes

Guarantee: Writes from same user are applied in order

Example: Database updates
T0: User writes: counter = 1
T1: User writes: counter = 2
T2: All replicas eventually see: 1 → 2 (not 2 → 1)

Implementation:
- Version numbers or vector clocks
- Reject out-of-order writes

Conflict Resolution

When replicas diverge, how do we resolve conflicts?

1. Last-Write-Wins (LWW)

Simplest approach: Highest timestamp wins

Example:
Replica A: Set value=42 at T=1000
Replica B: Set value=99 at T=1001

Resolution: value=99 (T=1001 is later)

Problem: Lost update! value=42 is discarded
Use case: Shopping cart (losing an item is acceptable)

2. Vector Clocks

Track causality to detect concurrent writes

Example:
Node A: [A:1, B:0] = "Hello"
Node B: [A:0, B:1] = "World"

Resolution: Concurrent writes detected! [A:1, B:1]
Action: Application decides (merge, prompt user, etc.)

Use case: DynamoDB, Riak (preserve both versions)

3. CRDTs (Conflict-Free Replicated Data Types)

Data structures that merge deterministically

Example: Counter CRDT
Replica A: increment → {A:1, B:0} = 1
Replica B: increment → {A:0, B:1} = 1

Merge: {A:1, B:1} = 2 (both increments preserved)

Use case: Collaborative editing (Google Docs), Redis CRDTs

Real Systems Using Eventual Consistency

System	Consistency Model	Conflict Resolution	Use Case
DynamoDB	Eventual (default)	Last-Write-Wins or Vector Clocks	High-availability key-value store
Cassandra	Tunable (eventual default)	Last-Write-Wins with timestamps	Multi-datacenter database
DNS	Eventual (TTL-based)	Authoritative server wins	Global name resolution
S3	Eventual (new objects immediate)	Latest timestamp wins	Object storage
Riak	Eventual (with siblings)	Vector clocks + application merge	Shopping carts

Case Study: DynamoDB

DynamoDB Read Levels:

Eventually Consistent Read (default):
- Reads from any replica
- May return stale data
- Latency: ~10ms
- Throughput: 2x strongly consistent reads
- Cost: 50% cheaper

Strongly Consistent Read (optional):
- Reads from primary/leader replica
- Always returns latest data
- Latency: ~20ms
- Throughput: Half of eventually consistent
- Cost: 2x more expensive

Use Case Decision:
- User profile reads: Eventually consistent ✓ (stale profile OK)
- Bank balance: Strongly consistent ✓ (need accurate balance)
- Product inventory: Eventually consistent ✓ (slight oversell acceptable)

When to Use Eventual Consistency

✓ Perfect Use Cases

High Availability Requirements

Scenario: Global DNS service (Route53, Cloudflare)
Requirement: Cannot afford downtime during network partition
Choice: Eventual consistency (AP)
Trade-off: Stale DNS records for <60 seconds OK

Multi-Region Systems

Scenario: Social media posts across continents
Requirement: Low latency in all regions
Choice: Eventual consistency with multi-master writes
Trade-off: Posts may appear in different order briefly

High Write Throughput

Scenario: IoT sensor data collection
Requirement: Ingest millions of writes/second
Choice: Eventual consistency (no sync blocking)
Trade-off: Sensor readings may arrive out of order

Tolerates Temporary Inconsistency

Scenario: E-commerce product catalog
Requirement: Fast page loads more important than perfect accuracy
Choice: Eventual consistency with cache
Trade-off: Price changes may take 30 seconds to propagate

✕ When NOT to Use

Financial Transactions

Problem: Cannot have conflicting bank balances
Example: ATM withdrawal + online purchase = overdraft
Solution: Strong consistency (CP) or distributed transactions

Inventory Management

Problem: Overselling products due to stale reads
Example: 1 item left, 2 users see "available", both buy
Solution: Strong consistency with pessimistic locking

Strong Ordering Requirements

Problem: Events must be processed in exact order
Example: Database replication log (apply ops in order)
Solution: Strong consistency or total order broadcast

Interview Application

Common Interview Question

Q: “Design a shopping cart for an e-commerce site. Should you use eventual or strong consistency?”

Strong Answer:

“I’d use eventual consistency for the shopping cart. Here’s my reasoning:

Why Eventual Consistency:

1. Availability matters: Users should always be able to add items, even during network issues
2. Temporary inconsistency is acceptable: If a user adds an item and it takes 200ms to propagate to all replicas, that’s fine
3. Multi-region performance: Low latency in all geographic regions
4. Conflict resolution is straightforward: Cart operations are mostly additive

Conflict Resolution Strategy:

• Use Last-Write-Wins for item quantities
• Alternatively, use CRDT (addition-based counter) to merge carts
• If two regions add the same item concurrently, merge by summing quantities

Strong Consistency for Checkout:

• Switch to strong consistency during payment
• Verify inventory availability with strong read
• Deduct inventory with ACID transaction

Real-World Example:

• Amazon uses eventual consistency for carts (famously described in Dynamo paper)
• Better to show users an item briefly, then say ‘out of stock’ at checkout, than to have site downtime”

Why This Answer Works:

• Identifies appropriate consistency model with reasoning
• Explains trade-offs clearly
• Discusses conflict resolution strategy
• Mentions when to switch to strong consistency (checkout)
• References real-world implementation (Amazon)

Code Example

Eventually Consistent Read in DynamoDB (Node.js)

const AWS = require("aws-sdk");
const dynamodb = new AWS.DynamoDB.DocumentClient();

// Eventually Consistent Read (default, faster, cheaper)
async function getEventuallyConsistent(userId) {
  const params = {
    TableName: "Users",
    Key: { userId: userId },
    ConsistentRead: false, // Eventually consistent (default)
  };

  const result = await dynamodb.get(params).promise();

  // May return stale data if recent write occurred
  // Typical lag: 10-50ms
  // Use case: Display user profile, product listings

  return result.Item;
}

// Strongly Consistent Read (slower, more expensive)
async function getStronglyConsistent(userId) {
  const params = {
    TableName: "Users",
    Key: { userId: userId },
    ConsistentRead: true, // Strongly consistent
  };

  const result = await dynamodb.get(params).promise();

  // Always returns latest data
  // Typical lag: 0ms
  // Use case: Bank balance, inventory checks before purchase

  return result.Item;
}

// Example: Shopping cart with eventual consistency
async function addToCart(userId, itemId, quantity) {
  // Write is always strongly consistent in DynamoDB
  // (single-item writes are ACID)
  await dynamodb
    .put({
      TableName: "ShoppingCarts",
      Item: {
        userId: userId,
        itemId: itemId,
        quantity: quantity,
        updatedAt: Date.now(),
      },
    })
    .promise();

  // Subsequent reads may be eventually consistent
  // Read-your-writes: Route user to same replica or use consistent read

  // Read cart (eventually consistent, faster)
  const cart = await dynamodb
    .query({
      TableName: "ShoppingCarts",
      KeyConditionExpression: "userId = :userId",
      ExpressionAttributeValues: { ":userId": userId },
      ConsistentRead: false, // Allow stale reads (< 1 second lag)
    })
    .promise();

  return cart.Items;
}

// At checkout: Switch to strong consistency
async function checkout(userId) {
  // Critical operation: Need accurate inventory and cart

  const cart = await dynamodb
    .query({
      TableName: "ShoppingCarts",
      KeyConditionExpression: "userId = :userId",
      ExpressionAttributeValues: { ":userId": userId },
      ConsistentRead: true, // Strong consistency required!
    })
    .promise();

  // Check inventory with strong consistency
  for (const item of cart.Items) {
    const product = await dynamodb
      .get({
        TableName: "Inventory",
        Key: { productId: item.itemId },
        ConsistentRead: true, // Must be accurate!
      })
      .promise();

    if (product.Item.stock < item.quantity) {
      throw new Error(`Insufficient stock for ${item.itemId}`);
    }
  }

  // Proceed with payment (ACID transaction)
  // ...
}

Related Content

Prerequisites:

• Distributed Systems Basics - Foundation concepts

Related Concepts:

• Leader-Follower Replication - How replication works
• Consensus - Strong consistency alternative
• Quorum - Tunable consistency

Used In Systems:

• Shopping cart systems (Amazon, e-commerce)
• Social media feeds (Twitter, Facebook)
• CDN and caching (Cloudflare, Akamai)

Explained In Detail:

• Distributed Systems Deep Dive - Consistency models in depth

Quick Self-Check

• Can explain eventual consistency in 60 seconds?
• Can draw timeline showing write propagation delay?
• Understand CAP theorem trade-off (AP vs CP)?
• Can name 3 real systems using eventual consistency?
• Know when to choose eventual vs strong consistency?
• Understand conflict resolution strategies (LWW, vector clocks, CRDTs)?