Idempotence

Operations that produce the same result when applied multiple times, critical for reliable distributed systems with retries and duplicate message handling

TL;DR

Idempotent operations produce the same result regardless of how many times they’re executed. In distributed systems, idempotence enables safe retries, duplicate message handling, and effectively-once semantics (at-least-once delivery + idempotent processing) without complex deduplication logic. Critical for building reliable APIs and message processing systems.

Visual Overview

Idempotence Overview
NON-IDEMPOTENT OPERATION (Dangerous with retries)
┌────────────────────────────────────────────────┐
│  Operation: account.balance += 100 (INCREMENT) │
│                                                │
│  Execution 1: balance = 1000 + 100 = 1100 ✓    │
│  Network fails, client retries...              │
│  Execution 2: balance = 1100 + 100 = 1200 ✕    │
│  (WRONG! User charged twice)                   │
│                                                │
│  Problem: Multiple executions → different      │
│           results (unintended side effects)    │
└────────────────────────────────────────────────┘

IDEMPOTENT OPERATION (Safe with retries)
┌────────────────────────────────────────────────┐
│ Operation: account.balance = 1100 (SET)        │
│                                                │
│ Execution 1: balance = 1100 ✓                  │
│ Network fails, client retries...               │
│ Execution 2: balance = 1100 ✓                  │
│ (Same result! Safe)                            │
│                                                │
│ Property: Multiple executions → same result    │
└────────────────────────────────────────────────┘

IDEMPOTENT WITH REQUEST ID (Best Practice)
┌────────────────────────────────────────────────┐
│ Request: {                                     │
│ request_id: "txn_abc123",                      │
│ action: "deposit",                             │
│ amount: 100                                    │
│ }                                              │
│ ↓                                              │
│ Execution 1:                                   │
│ - Check: request_id processed? NO              │
│ - Execute: balance += 100 → 1100               │
│ - Store: request_id = "txn_abc123"             │
│ - Return: success ✓                            │
│ ↓                                              │
│ Network fails, client retries...               │
│ ↓                                              │
│ Execution 2:                                   │
│ - Check: request_id processed? YES             │
│ - Skip execution (already done)                │
│ - Return: success ✓ (same result)              │
│                                                │
│ Result: Safe retries, no duplicate processing  │
└────────────────────────────────────────────────┘

HTTP METHOD IDEMPOTENCE
┌────────────────────────────────────────────────┐
│ GET /users/123                                 │
│ → Idempotent ✓ (read, no side effects)         │
│                                                │
│ PUT /users/123 {"name": "Alice"}               │
│ → Idempotent ✓ (set to specific value)         │
│                                                │
│ DELETE /users/123                              │
│ → Idempotent ✓ (deleted or already deleted)    │
│                                                │
│ POST /users {"name": "Bob"}                    │
│ → NOT Idempotent ✕ (creates new resource)      │
│                                                │
│ POST /orders/123/pay                           │
│ → NOT Idempotent ✕ (charges money)             │
│ Unless: Use idempotency key                    │
└────────────────────────────────────────────────┘

Core Explanation

What is Idempotence?

Idempotence (from mathematics) means an operation produces the same result when applied multiple times:

Idempotence Definition
f(f(x)) = f(x)

Examples:

- SET value = 42: Idempotent (repeated SET has same effect)
- INCREMENT value: NOT idempotent (repeated INCREMENT increases value)
- DELETE item: Idempotent (item deleted, stays deleted)
- CREATE item: NOT idempotent (creates duplicate items)

In Distributed Systems:

Idempotence enables safe retries when network failures or timeouts occur, eliminating the need to distinguish between:

  • Request failed (retry needed)
  • Request succeeded but response lost (retry causes duplicate)

Why Idempotence Matters

Problem: Network Timeouts

Network Timeout Problem
Scenario: Payment API call

Client sends: "Charge $100 to card"
↓
Network timeout (no response received)
↓
Did payment succeed or fail?

Without idempotence:

- Don't retry → User may not be charged (bad)
- Retry → User may be charged twice (worse!)

With idempotence:

- Retry safely → charged effectively-once (retries collapse if any attempt succeeds) ✓

At-Least-Once + Idempotence = Exactly-Once

Message Delivery Guarantees
At-Most-Once:

- Message delivered 0 or 1 times
- May lose messages
- Use case: Metrics (OK to lose)

At-Least-Once:

- Message delivered 1+ times
- No loss, but duplicates possible
- Use case: Most systems

Exactly-Once:

- Message delivered exactly 1 time
- Hard to implement (requires transactions)
- OR: At-least-once + idempotent processing ✓

Idempotent vs Non-Idempotent Operations

Naturally Idempotent Operations:

Naturally Idempotent Operations
SET operations:
account.balance = 1100           ✓ Idempotent
user.email = "alice@example.com" ✓ Idempotent

DELETE operations:
DELETE FROM users WHERE id=123 ✓ Idempotent
(second delete: already gone)

Absolute updates:
UPDATE users SET status='active' ✓ Idempotent
WHERE id=123

Read operations:
SELECT * FROM users WHERE id=123 ✓ Idempotent
(no side effects)

NOT Idempotent (Require Special Handling):

Non-Idempotent Operations
INCREMENT operations:
account.balance += 100           ✕ Not idempotent
view_count++                     ✕ Not idempotent

CREATE operations:
INSERT INTO orders (id, amount) ✕ Not idempotent
(creates duplicate rows)

Relative updates:
UPDATE users SET age = age + 1 ✕ Not idempotent

Implementing Idempotence

1. Unique Request IDs (Idempotency Keys)

Idempotency Keys Implementation
API Request with idempotency key:

POST /api/payments
Headers:
Idempotency-Key: req_abc123xyz

Body:
{
"amount": 100,
"currency": "USD",
"card_id": "card_789"
}

Server-side implementation:
┌────────────────────────────────────────────┐
│ 1. Extract idempotency key from header     │
│ 2. Check if key exists in database:        │
│ - EXISTS: Return cached response ✓         │
│ - NOT EXISTS: Process request              │
│ 3. Execute business logic                  │
│ 4. Store key + response in database        │
│ 5. Return response                         │
└────────────────────────────────────────────┘

Result:

- First request (key=req_abc123xyz): Process payment
- Retry (same key): Return cached result, no duplicate charge
- New request (key=req_def456uvw): Process new payment

2. Natural Idempotency (Design for It)

Natural Idempotency Patterns
BAD (Non-Idempotent):
POST /orders
{
"product": "laptop",
"quantity": 1
}
→ Creates new order each time ✕

GOOD (Idempotent with client-generated ID):
PUT /orders/order_abc123
{
"product": "laptop",
"quantity": 1
}
→ Creates or updates order_abc123 ✓
→ Retry-safe

GOOD (Idempotent with unique constraint):
POST /orders
{
"client_order_id": "order_abc123", // Unique!
"product": "laptop",
"quantity": 1
}
→ Database unique constraint prevents duplicates ✓

3. Database-Level Idempotency

Database-Level Idempotency
Using database constraints:

CREATE TABLE payments (
id SERIAL PRIMARY KEY,
request_id VARCHAR(255) UNIQUE, -- Idempotency key
amount DECIMAL(10, 2),
status VARCHAR(50),
created_at TIMESTAMP
);

Application code:
INSERT INTO payments (request_id, amount, status)
VALUES ('req_abc123', 100.00, 'completed')
ON CONFLICT (request_id) DO NOTHING;

Result:

- First insert: Creates payment
- Retry: Conflict detected, no duplicate payment ✓

4. State Machine Approach

State Machine Idempotency
Payment state machine:

States: PENDING → PROCESSING → COMPLETED
↓
FAILED

Transitions are idempotent:

- PENDING → PROCESSING: OK
- PROCESSING → PROCESSING: OK (retry, same state)
- PROCESSING → COMPLETED: OK
- COMPLETED → COMPLETED: OK (already completed)

Implementation:
UPDATE payments
SET status = 'COMPLETED'
WHERE id = 123 AND status IN ('PENDING', 'PROCESSING')

→ Retrying "complete payment" is safe ✓

Idempotent Message Processing

Kafka Consumer Example:

Idempotent Kafka Consumer
Problem: Kafka guarantees at-least-once delivery
→ Messages may be processed multiple times

Solution: Idempotent consumer

Non-Idempotent Consumer (Bad):
┌────────────────────────────────────────────┐
│ consume message: "increment counter"       │
│ counter++ // ✕ Not idempotent              │
│ commit offset                              │
│                                            │
│ If crash before commit:                    │
│ → Reprocess message                        │
│ → counter++ again (duplicate)              │
└────────────────────────────────────────────┘

Idempotent Consumer (Good):
┌────────────────────────────────────────────┐
│ consume message: {                         │
│ message_id: "msg_123",                     │
│ action: "increment_counter"                │
│ }                                          │
│ ↓                                          │
│ if not processed(message_id):              │
│ counter++                                  │
│ mark_processed(message_id)                 │
│ ↓                                          │
│ commit offset                              │
│                                            │
│ If crash and reprocess:                    │
│ → Check: msg_123 processed? YES            │
│ → Skip increment ✓                         │
└────────────────────────────────────────────┘

Common Patterns

1. Stripe API Style

Stripe API Idempotency
POST /v1/charges
Headers:
Idempotency-Key: unique_key_here

Body:
{
"amount": 2000,
"currency": "usd"
}

Behavior:

- First request: Create charge, return 200
- Retry with same key: server state unchanged (charge ran once);
HTTP code is the cached original 200, not a fresh execution
- Different key: Create new charge
- Key expires after 24 hours

2. AWS S3 Style

AWS S3 Idempotency
PUT /bucket/object.txt
Content: "Hello World"

Behavior:

- Uploading same object multiple times: Idempotent ✓
- Result always: object.txt contains "Hello World"
- Uses content-based addressing (ETag)

3. Database Upsert Style

Database Upsert Idempotency
INSERT INTO users (id, name, email)
VALUES (123, 'Alice', 'alice@example.com')
ON DUPLICATE KEY UPDATE
name = VALUES(name),
email = VALUES(email)

Behavior:

- First call: Insert new user
- Retry: Update user (same result)
- Idempotent ✓

Real Systems Using Idempotence

SystemIdempotency MechanismKey FeatureUse Case
Stripe APIIdempotency-Key header24-hour key expirationPayment processing
AWS APIsClient request tokenService-specificCloudFormation, EC2
KafkaMessage offset + deduplicationConsumer-sideStream processing
KubernetesDeclarative desired stateReconciliation loopContainer orchestration
HTTP PUTResource URIREST semanticsRESTful APIs
GitContent-addressableSHA hashesVersion control

Case Study: Stripe Payments

Stripe Idempotency Implementation
POST https://api.stripe.com/v1/charges
Headers:
Authorization: Bearer sk*test*...
Idempotency-Key: req_abc123

Body:
amount=2000&currency=usd

First Request:

1. Server checks: key "req_abc123" exists? NO
2. Process payment → charge card
3. Store: {key: "req_abc123", response: {...}, ttl: 24h}
4. Return: 200 OK {id: "ch_789", amount: 2000}

Retry (network timeout):

1. Server checks: key "req_abc123" exists? YES
2. Fetch cached response
3. Return: 200 OK {id: "ch_789", amount: 2000}
 (Same charge ID, no duplicate payment)

Different Request:
Idempotency-Key: req_def456
→ New payment, different charge ID

Key Expiration:

- Keys expire after 24 hours
- After expiration, same key creates new charge

Case Study: Kafka Idempotent Producer

Kafka Idempotent Producer
Kafka Producer Idempotence (since 0.11):

Properties config = new Properties();
config.put("enable.idempotence", "true");
config.put("acks", "all");
config.put("retries", Integer.MAX_VALUE);

How it works:
┌────────────────────────────────────────────┐
│ Producer assigns sequence numbers:         │
│ Message 1: {seq: 0, data: "msg1"}          │
│ Message 2: {seq: 1, data: "msg2"}          │
│ Message 3: {seq: 2, data: "msg3"}          │
│ ↓                                          │
│ Broker tracks: producer_id + seq number    │
│ ↓                                          │
│ If duplicate received:                     │
│ - Message seq=1 already written            │
│ - Discard duplicate, ACK success ✓         │
│ ↓                                          │
│ Result: exactly-once delivery per          │
│         partition (no dupes for that        │
│         producer_id + seq)                  │
└────────────────────────────────────────────┘

Guarantees:
✓ No duplicate messages in partition
✓ Messages ordered within partition
✓ Safe retries (producer can retry forever)

When to Use Idempotence

✓ Perfect Use Cases

Payment Processing

Payment Processing Use Case
Scenario: Credit card charges
Requirement: Never double-charge users
Solution: Idempotency keys for payment API
Benefit: Safe retries on network failures

Order Processing

Order Processing Use Case
Scenario: E-commerce order placement
Requirement: Same order submitted multiple times → single order
Solution: Client-generated order ID
Benefit: Prevent duplicate orders

Inventory Updates

Inventory Updates Use Case
Scenario: Deduct inventory on purchase
Requirement: Don't deduct twice on retry
Solution: Transaction ID + database constraint
Benefit: Accurate inventory counts

Message Processing

Message Processing Use Case
Scenario: Kafka consumer processing events
Requirement: Each message has effect exactly once (effectively-once)
Solution: Message ID tracking
Benefit: At-least-once + idempotence = exactly-once

✕ When NOT to Use (or Use Carefully)

CaseExampleConsiderationRecommendation
Intentional DuplicatesUser clicking “Add to Cart” multiple timesIntent is to add multiple itemsDon’t apply idempotency for this use case
Time-Sensitive OperationsStock trading (buy at current price)Price changes between retriesIdempotency key + timestamp validation
Analytics/MetricsPage view countersSlight overcounting on retries is acceptableUse approximate counters (HyperLogLog)

Interview Application

Common Interview Question

Q: “Design an API for a payment system. How would you handle network retries to prevent double-charging users?”

Strong Answer:

“I’d implement idempotent payment processing using idempotency keys:

API Design:

POST /api/v1/payments
Headers:
  Authorization: Bearer token
  Idempotency-Key: unique_request_id
Body:
  {
    "amount": 100.00,
    "currency": "USD",
    "payment_method_id": "pm_123"
  }

Server-Side Implementation:

  1. Extract Idempotency Key:

    • Required header, client-generated UUID
    • Example: Idempotency-Key: req_a1b2c3d4

  2. Check Idempotency Table:

    CREATE TABLE idempotency_keys (
  key VARCHAR(255) PRIMARY KEY,
  request_hash VARCHAR(255),
  response_status INT,
  response_body TEXT,
  created_at TIMESTAMP,
  INDEX idx_created (created_at)
);

  3. Processing Logic:

    BEGIN TRANSACTION
  SELECT * FROM idempotency_keys WHERE key = :key

  IF EXISTS:
    // Validate request unchanged (hash matches)
    IF request_hash matches:
      RETURN cached response ✓
    ELSE:
      RETURN 400 Bad Request (key reused with different request)

  ELSE:
    // First time seeing this key
    // Process payment
    charge = stripe.charges.create(...)

    // Store idempotency record
    INSERT INTO idempotency_keys (
      key, request_hash, response_status, response_body
    ) VALUES (:key, :hash, 200, :response)

    COMMIT TRANSACTION
    RETURN 200 OK {charge_id: ...}

Benefits:

This gives the client safe retries and prevents double charging because the same key returns the same result. The request hash keeps the key honest: if a caller reuses a key with a different payload, the API rejects it instead of silently replaying the wrong response.

Key Management:

Keys need an operational lifecycle. I would expire them after 24 hours, remove old records with a cleanup job, and alert on a high duplicate rate because that can signal client retry storms or misuse.

Edge Cases:

Concurrent requests with the same key should serialize through database locking, such as SELECT FOR UPDATE, so the first request processes and the rest wait for the stored result. Partial failures are handled by storing the idempotency key in the payment record itself, which lets recovery find the charge even if the idempotency row write failed. Key reuse is rejected when the stored request hash does not match the incoming payload.

Alternatives Considered:

No idempotency is unacceptable because it can double-charge a user. Request deduplication alone is not enough because callers need the same response back, not just suppression. A distributed lock would work, but the database-backed key is simpler and keeps the correctness boundary close to the payment write.

Real-World Example: Stripe uses this exact pattern with Idempotency-Key header”

Code Example

Idempotent Payment API

from flask import Flask, request, jsonify
import hashlib
import json
import uuid
from datetime import datetime, timedelta

app = Flask(__name__)

# Simple in-memory store (use database in production)
idempotency_store = {}
payments_store = {}

def compute_request_hash(request_data):
    """Compute hash of request body for validation"""
    return hashlib.sha256(
        json.dumps(request_data, sort_keys=True).encode()
    ).hexdigest()

@app.route('/api/v1/payments', methods=['POST'])
def create_payment():
    """Idempotent payment endpoint"""

    # ... omitted: keep concept snippets short
    print(f"Cleaned up {len(keys_to_remove)} old idempotency keys")

if __name__ == '__main__':
    # Example usage:
    # curl -X POST http://localhost:5000/api/v1/payments \
    #   -H "Content-Type: application/json" \
    #   -H "Idempotency-Key: req_abc123" \
    #   -d '{"amount": 100.00, "currency": "USD"}'

    app.run(debug=True, port=5000)

Idempotent Kafka Consumer

import json
from kafka import KafkaConsumer

class IdempotentConsumer:
    """Kafka consumer with idempotent message processing"""

    def __init__(self, topic, processed_messages_store):
        self.consumer = KafkaConsumer(
            topic,
            bootstrap_servers=['localhost:9092'],
            enable_auto_commit=False,  # Manual commit after processing
            value_deserializer=lambda m: json.loads(m.decode('utf-8'))
        )
        self.processed_messages = processed_messages_store

    def process_messages(self):
        """Process messages idempotently"""
        for message in self.consumer:
            # Extract message ID (required for idempotence)
            msg_data = message.value
            message_id = msg_data.get('message_id')

    # ... omitted: keep concept snippets short

        elif action == 'send_email':
            recipient = msg_data['recipient']
            # Send email...
            print(f"Sending email to: {recipient}")

# Usage
processed_messages = set()  # In production: Use Redis/DB
consumer = IdempotentConsumer('events', processed_messages)
consumer.process_messages()

Related Content

Prerequisites:

Related Concepts:

Used In Systems:

  • Stripe API: Idempotency keys for payments
  • Kafka: Idempotent producer and consumer patterns
  • REST APIs: HTTP PUT/DELETE idempotent semantics

Explained In Detail:

  • Distributed Systems Deep Dive - Idempotence patterns

Quick Self-Check

  • Can explain idempotence in 60 seconds?
  • Know difference between idempotent and non-idempotent operations?
  • Understand how idempotency keys work?
  • Can implement idempotent API endpoint?
  • Know how at-least-once + idempotence = exactly-once?
  • Can design idempotent message processing?

Production signal

Why this concept matters

Interview 70% of API design interviews
Production Payment systems, order processing
Performance Safe retries
Scale At-least-once + idempotence