What Is Dual-Write Problem?

What is the dual-write problem?

The problem is structural, not a coding error. Heterogeneous systems — a relational database, Kafka, Redis, Elasticsearch — do not share a transaction coordinator, so there is no atomic operation covering a write to two of them. Two-phase commit would provide one in principle, but most brokers, caches, and search engines do not participate in XA-style protocols, and 2PC brings blocking and availability costs that modern infrastructure teams generally decline.

Dual writes appear wherever one fact must live in two places: update the database and publish an event; update the database and invalidate the cache; update the database and reindex the document. Each pairing has the same flaw, and no amount of error handling in application code removes it — retries create duplicates, and reordering the writes only changes which system ends up wrong.

How the dual-write problem works

Consider a service that commits an order to Postgres, then publishes to Kafka:

1. 1.BEGIN; INSERT INTO orders ...; COMMIT; — succeeds.
2. 2.The process crashes (or the broker is unreachable) before the publish.
3. 3.The order exists, but no event is ever emitted. Fraud scoring, inventory, and the search index never learn about it.

Reversing the order flips the failure rather than fixing it:

1. 1.Publish OrderPlaced to Kafka — succeeds; consumers begin processing.
2. 2.The database INSERT fails or the process dies before commit.
3. 3.Downstream systems now act on a phantom event for an order that does not exist.

Even without crashes, concurrency bites: two writers updating the same entity can interleave so that the database applies the writes in one order while the broker records events in the other, leaving downstream state permanently different from the source even though every individual write "succeeded."

Why the dual-write problem matters in real-time systems

Dual writes are the root cause of drift between a system of record and its derived stores. Every cache, index, feature snapshot, and per-service copy that is maintained by a second, independent write is one lost update away from diverging — and each copy drifts independently, so two services consulting two copies can hold different versions of the same fact at the same moment. This is the failure mode underlying much of what goes wrong in polyglot persistence architectures, where a decision assembled from several stores has no guarantee those stores agree.

The accepted resolutions all reduce two writes to one. The outbox pattern folds the event into the database transaction and relays it afterward. Change data capture goes further: the application writes only to the database, and the event stream is derived from the database's own transaction log, so the stream reflects exactly the committed changes, in commit order. Event sourcing makes the log itself the single write target. In every case the fix is the same shape — one authoritative write, with all other representations derived asynchronously from it.

FAQ