A consumer reads a payment event, updates a database, then commits its Kafka offset. The process dies after the database commit and before the offset commit. On restart, Kafka delivers the event again.
Turning on a transactional producer does not roll back the first database update. There are two commit boundaries, and the failure happened between them.
Name the effects inside the transaction
Kafka can atomically commit output records to Kafka topics together with the consumed offsets. Downstream consumers using read_committed skip aborted transactional output. Processing code can execute again after an abort; the guarantee concerns committed Kafka effects, not the number of times a Java method runs.
The Kafka design documentation makes the external-system boundary explicit: stronger guarantees for another destination require that system’s cooperation. A database, an email provider and a payment API do not join a Kafka transaction just because their calls sit between beginTransaction() and commitTransaction().
| Failure point | What can survive | Recovery obligation |
|---|---|---|
| Before database commit | Neither database effect nor offset | Retry safely |
| After database commit, before offset commit | Database effect only | Detect replay without applying it twice |
| After both commits | Both | Continue from the committed offset |
Committing the offset first changes the failure mode: a crash can skip an effect that never happened. Swapping the order does not remove the gap.
Put duplicate detection beside the effect
For a database sink, one approach is an inbox table. In a single database transaction, insert the input’s durable identity under a uniqueness constraint and apply its business effect only if the insert succeeded. Commit the Kafka offset after that transaction succeeds.
The identity might be a source-system event ID, or a tuple containing the source cluster/topic identity, partition and offset. Choose deliberately. An offset identifies a record in a particular log; it may not identify the same business operation republished into another topic.
The important atomic unit is:
BEGIN DATABASE TRANSACTION
insert input identity if absent
if inserted: apply business update
COMMIT DATABASE TRANSACTION
commit Kafka offsetDo not implement this as “SELECT whether seen, then later INSERT.” Concurrent handlers can both observe absence. The database constraint and transaction must arbitrate. Do not advance an offset past earlier unfinished records when processing a partition concurrently.
The runnable fixture deliberately applies one synthetic event twice. A naive handler increments the balance twice. A SQLite transaction combining an inbox insert with the balance update applies it once. This demonstrates the local atomicity pattern; it does not test Kafka fencing, rebalances, or a production database driver.
An outbox solves a different half
When the database originates an event, write the business row and an outbox row in the same database transaction. A relay publishes the outbox later.
If the relay marks the row published before Kafka commits, it can lose the event. If it marks the row after Kafka commits, a crash in between can cause publication again. The second ordering supports retry, but downstream consumers must handle duplicate event IDs. A Kafka transaction around the publish does not atomically include the database’s “published” flag.
The relay also needs ownership of pending rows, retry tracking, and a policy for poison events. These are separate from whether the original business transaction successfully created the outbox entry.
The test is a restart
Kill the handler after the destination commits and before the source progress is recorded. Restart it. Assert the destination state and the consumed position independently. Repeat with concurrent delivery, a conflicting payload under the same ID, and expired deduplication records.
For an external API, use its documented idempotency contract if one exists. Keep the key stable across retries and check its retention period. An inbox transaction in your own database cannot make a remote charge atomic.
Read the updated Kafka transactions chapter for the Kafka-side mechanics and safe retries for the broader application boundary. The useful question in a design review is: which durable effect and which progress marker commit together?