Path 5 · lesson 4 of 5
Kafka transactions and your database
A Kafka transaction covers Kafka partitions and consumer offsets, never your database. See the dual-write failure windows and the designs that close them.
What you will learn
- Name the two failure windows of a dual write and say what the database and the topic contain after a crash in each
- Explain why a Kafka transaction and a JDBC transaction cannot be committed as one atomic unit
- Describe exactly what Spring's transaction synchronisation does, in which order it commits, and which window remains
- Choose between a transactional outbox, change data capture, offsets stored in the database and an idempotent consumer for a given service
- Read a payment or order incident and decide which side of the boundary lost or duplicated the effect
Before you start
- Delivery guarantees
- Duplicates and idempotency
- Kafka transactions
- How a relational database transaction commits and rolls back
The concept
An order service does two things when a customer checks out: it inserts a row into its orders table, and it publishes OrderPlaced so that payment, inventory and notifications can react. Those are writes to two different systems. The database commits when the database says so. Kafka accepts the record when the partition leader says so. Nothing ties the two decisions together, so the service can die after one and before the other. This is the dual-write problem.
It is tempting to reach for Kafka transactions here, because the word is the same. A Kafka transaction makes a set of writes to Kafka partitions, plus the consumer offsets sent through the producer, commit or abort together. That is the whole of its scope. A row in PostgreSQL, a charge at a payment provider and an email are outside it, and no client setting moves them inside.
| What you want to be atomic | Covered by a Kafka transaction? | What covers it instead |
|---|---|---|
| Records written to several topic partitions | Yes | The transactional producer |
| Those records plus the offsets of the input you consumed | Yes | sendOffsetsToTransaction (Spring's listener container calls it for you) |
An INSERT into orders plus the OrderPlaced record | No | Outbox or CDC: one database transaction, publishing derived from it |
A consumed record's offset plus an UPDATE in your database | No | Idempotent write, or the offset stored in the same database transaction |
| A card charge at a provider plus anything else | No | An idempotency key the provider honours, and a retry |
Why it exists
Take the two possible orders of a dual write. In both, the code looks correct and passes every test, because tests rarely kill the process between two lines.
| Order of the two writes | Crash or failure here | Database | Kafka | What the business sees |
|---|---|---|---|---|
| Database commit, then send | After the commit, before the send is acknowledged | Order 7001 exists | No OrderPlaced for 7001 | A paid-for basket that is never charged, never reserved, never shipped. Nothing retries it, because the request already returned or died. |
| Database commit, then send | The send fails after the commit (broker unavailable, delivery.timeout.ms expires) | Order 7001 exists | No OrderPlaced for 7001 | Same as above, and the caller may also receive an error for an order that was in fact saved. |
| Send, then database commit | After the send is acknowledged, before the commit | No order 7002 | OrderPlaced for 7002 is in the log | The payment service charges a card for an order that does not exist. Support cannot find it. |
| Send, then database commit | The commit fails (constraint violation, deadlock victim, lost connection) | No order 7002 | OrderPlaced for 7002 is in the log | A phantom event. You cannot take a record back out of a partition. |
| Either order | The send times out but the broker did append the record | Depends on what the code does with the exception | Record is there | Code that rolls the order back on a send error has now produced a phantom event from a different direction. |
Before patterns such as the outbox became routine, teams lived with these windows and ran reconciliation jobs to find orders without events. The windows are narrow, so the bug shows up as a handful of stuck orders per month, usually during a deploy or a broker restart, which is exactly when many processes are being stopped between two lines.
Outbox pattern
Simulation, not a real clusterThe animated Build lesson shows the same dual write failing and the outbox closing the window. Simulated, not a real broker.
Step 1 · The pattern
Save it, then tell everyone
An order comes in. The service saves it to its database, then publishes an event so shipping, email and analytics can react. Two writes, to two systems.
- order request
- done
How it works inside
Why Kafka cannot enlist your database
Inside Kafka, a transaction is run by a transaction coordinator, a broker that records each transaction's state in an internal log. The producer registers the partitions it writes to, and on commit the coordinator first records the decision durably and then writes commit markers into every partition involved, including the consumer offsets partition when offsets were sent. That is a two-phase protocol, but every participant is a Kafka partition. KIP-98, which introduced it, defines no step in which an outside resource can be asked to prepare and then told the outcome.
A relational database has its own commit protocol and its own log. To commit both atomically you would need a coordinator that both sides obey: each side would first promise it can commit (prepare), and only then be told to commit. The standard Kafka producer API offers beginTransaction, commitTransaction and abortTransaction, with no prepare step you can hand to a JTA or XA transaction manager. Spring therefore has two separate transaction managers for the two systems, and the best it can do is call them one after the other.
What Spring's transaction synchronisation gives you
With a transactional producer factory (a transaction-id-prefix is set) and a method annotated @Transactional that is bound to the database transaction manager, a KafkaTemplate.send inside the method starts a Kafka transaction and registers it with the database transaction. When the method returns, Spring commits the database transaction and then commits the Kafka transaction. If the method throws, both roll back. The Spring reference describes this as synchronisation; it is two commits in a fixed order, not one.
| Setup | Commit order | Window that remains | Result of a failure in the window |
|---|---|---|---|
Producer side, @Transactional on the database manager, template synchronises | Database, then Kafka | Database committed, Kafka commit fails or the process dies | Order row without an event. Current versions throw the failure to the caller, who must compensate; a dead process cannot. |
Producer side, nested @Transactional: outer database manager, inner KafkaTransactionManager | Kafka, then database | Kafka committed, database commit fails | Event without an order row. |
Consumer side: the listener container starts the Kafka transaction, the listener method is @Transactional on the database manager | Database, then offsets and Kafka | Database committed, Kafka commit fails or the process dies | Output records are aborted and the input record is delivered again, so the database write runs a second time. |
The third row is the useful one. A consumer has something the producer side lacks: the input record is still in the log, and its offset did not move, so the work is retried automatically. The Spring reference says as much: when the Kafka commit fails after the database commit, the record is redelivered, so the database update must be idempotent. Its chapter on exactly-once semantics adds that the read and the process steps keep at-least-once semantics; only the read-process-write sequence as a whole completes once in Kafka.
Where the Kafka transaction ends
Simulation · not a real clusterTransactional mode: offsets and output records commit together. Crash after processing and watch which effects are rolled back and which one is not.
The designs that actually close the window
| Design | How it removes the dual write | What you still have to handle |
|---|---|---|
| Transactional outbox | The event is a row inserted in the same database transaction as the business row. A relay publishes rows afterwards. | The relay publishes at least once, so consumers must tolerate duplicates. Ordering per key needs care in the relay. |
| Change data capture | A connector reads the database's own commit log and publishes what was committed, often from an outbox table. | Operating Kafka Connect and the connector. Still at-least-once towards consumers after a connector restart. |
| Offsets stored with the data | The consumer writes its result and the next offset to read in one database transaction, and seeks to the stored offset when a partition is assigned. | You own offset storage and seeking (ConsumerSeekAware in Spring). Kafka's committed offsets and lag tooling no longer describe the truth for that group. |
| Idempotent consumer | The write is safe to repeat: a unique key or a processed-events row in the same database transaction rejects the second attempt. | Every event needs a stable identity, and the dedupe record must be kept long enough. |
The Kafka design documentation points at the third design itself: when writing to an external system, let the consumer keep its position in the same place as its output, because many stores cannot take part in a two-phase commit. All four designs replace two independent commits by one commit in one system plus something that can be repeated safely.
Architecture
Drawing diagram…
A payment service writes to two boxes. One box, owned by Kafka's transaction coordinator, holds output records and consumer offsets. The other, owned by the database engine, holds the payments, outbox and processed-events rows. A card provider call sits outside both.
flowchart LR
IN["orders.order-placed"] --> APP["Payment Service"]
subgraph KTX["Kafka transaction"]
OUT["payments.payment-result records"]
OFF["Offsets of group payment-service"]
end
subgraph DBTX["Database transaction"]
PAY["payments row"]
OBX["outbox row"]
DED["processed_events row"]
end
APP --> OUT
APP --> OFF
APP --> PAY
APP --> OBX
APP --> DED
TC["Transaction coordinator on a broker"] --> OUT
TC --> OFF
DBE["Database engine and its log"] --> PAY
APP -.-> EXT["Card provider HTTP call, in neither transaction"]- The left box commits when the transaction coordinator writes commit markers. The right box commits when the database flushes its own log. Neither waits for the other.
- Everything inside one box is all-or-nothing. That is why the outbox row and the processed-events row live in the database box next to the business row: they borrow the database's atomicity.
- The provider call is in neither box. It can only be made safe by being repeatable, which the next lesson covers.
- A design is sound when a crash at any point leaves each box internally consistent and the next retry brings the other box up to date.
Message flow
Drawing diagram…
Sequence diagram with an order service, a database and Kafka. For order 7001 the database commits, the service crashes, and the event is never sent. For order 7002 the event is sent and acknowledged, the service crashes, and the database rolls back.
sequenceDiagram participant S as Order Service participant D as Orders DB participant K as Kafka Note over S,K: Order 7001, database first S->>D: BEGIN and INSERT order 7001 S->>D: COMMIT D-->>S: committed Note over S: crash here S-xK: send OrderPlaced 7001 never happens Note over D,K: 7001 is saved but no service hears about it Note over S,K: Order 7002, Kafka first S->>D: BEGIN and INSERT order 7002 S->>K: send OrderPlaced 7002 K-->>S: acknowledged Note over S: crash here S-xD: COMMIT never happens D-->>D: connection lost, rollback Note over D,K: 7002 is in the topic but not in the database
- Order 7001: the commit is durable, so restarting the service does not undo it. No code path remembers that an event is owed.
- Order 7002: the record is in the partition and consumers will read it. The database rolled back because the connection dropped before the commit.
- Wrapping the send in a Kafka transaction changes only the second half: an uncommitted record can be aborted. It moves the window to between the two commits; it does not remove it.
- With an outbox, the first sequence becomes safe: the owed event is itself a committed row, so a relay finds and publishes it after any restart.
In a real system
A typical checkout: the order service owns a PostgreSQL schema and publishes to orders.order-placed, 12 partitions, keyed by orderId so that every event about one order stays in one partition. The payment service consumes it in group payment-service, charges the card and publishes payments.payment-result, also keyed by orderId.
- Order service (producer side). It has no input record to be redelivered, so nothing repairs a lost send. It writes the order and an
outboxrow in one database transaction. A relay or a CDC connector publishes the row. See the lab Transactional outbox. - Payment service (consumer side). It is redelivered automatically after a crash, so it needs repeatable writes: a unique constraint on
order_idinpayments, and its own outbox row forpayments.payment-resultin the same database transaction as the status change. - Ledger projection (consumer only, no outgoing events). A consumer that builds a balance table can store
(topic, partition, next_offset)in the same transaction as each balance update and seek to it on assignment. A replayed record then lands on an offset the database has already passed and is skipped. - Fraud scoring (Kafka in, Kafka out, no database). This is the one place where a Kafka transaction alone is enough: read
payments.payment-result, writefraud.scores, commit offsets and output together.
Java and Spring Boot
The order service, first as the dual write to recognise in code review, then with an outbox. PostgreSQL syntax. The relay shown is the smallest correct one; the transactional outbox lesson covers ordering, batching and CDC.
package com.example.orders;import java.time.Instant;import java.util.UUID;import org.springframework.jdbc.core.JdbcTemplate;import org.springframework.kafka.core.KafkaTemplate;import org.springframework.transaction.annotation.Transactional;/** Do not copy: this is the dual write. Kept here so you can recognise it. */public class DualWriteOrderService { private final JdbcTemplate jdbc; private final KafkaTemplate<String, OrderPlaced> kafka; public DualWriteOrderService(JdbcTemplate jdbc, KafkaTemplate<String, OrderPlaced> kafka) { this.jdbc = jdbc; this.kafka = kafka; } @Transactional public UUID placeOrder(PlaceOrder cmd) { UUID orderId = UUID.randomUUID(); jdbc.update( "INSERT INTO orders (order_id, customer_id, amount_minor, currency, status) VALUES (?, ?, ?, ?, 'PLACED')", orderId, cmd.customerId(), cmd.amountMinor(), cmd.currency()); // The send leaves this method before the database commit below has happened. kafka.send("orders.order-placed", orderId.toString(), new OrderPlaced(UUID.randomUUID(), orderId, cmd.customerId(), cmd.amountMinor(), cmd.currency(), Instant.now())); return orderId; // The database commits here, after the method returns. // Commit fails -> the event is already on its way. Send fails later -> the order is already saved. }}Why it is written this way
sendis asynchronous: it returns a future once the record is in the producer's buffer. The method can return, and the database can commit, before the broker has accepted or rejected the record.- The
@Transactionalhere is bound to the database only. Without atransaction-id-prefixthe template is not transactional and takes no part in it at all. - Setting a
transaction-id-prefixwould make Spring commit the database first and Kafka second. The window between those two commits remains, and on this producer side nothing retries the request. - Waiting on the future inside the method does not help either: it only chooses which of the two failure windows you get, and it holds a database connection during a network call.
CREATE TABLE orders ( order_id uuid PRIMARY KEY, customer_id varchar(64) NOT NULL, amount_minor bigint NOT NULL, currency varchar(3) NOT NULL, status varchar(16) NOT NULL, created_at timestamptz NOT NULL DEFAULT now());CREATE TABLE outbox ( id bigserial PRIMARY KEY, event_id uuid NOT NULL UNIQUE, topic varchar(128) NOT NULL, record_key varchar(128) NOT NULL, payload text NOT NULL, created_at timestamptz NOT NULL DEFAULT now(), published_at timestamptz);CREATE INDEX outbox_unpublished ON outbox (id) WHERE published_at IS NULL;Why it is written this way
event_idis created once, with the business fact. Every later copy of the event, however many times the relay sends it, carries the same id, which is what consumers deduplicate on.record_keyholds the order id so the relay publishes with the same key the consumers expect for per-order ordering.- The partial index keeps the relay's query cheap when the table holds millions of published rows awaiting cleanup.
package com.example.orders;import java.time.Instant;import java.util.UUID;import com.fasterxml.jackson.core.JsonProcessingException;import com.fasterxml.jackson.databind.ObjectMapper;import org.springframework.jdbc.core.JdbcTemplate;import org.springframework.stereotype.Service;import org.springframework.transaction.annotation.Transactional;record PlaceOrder(String customerId, long amountMinor, String currency) {}record OrderPlaced(UUID eventId, UUID orderId, String customerId, long amountMinor, String currency, Instant occurredAt) {}@Servicepublic class OrderService { private final JdbcTemplate jdbc; private final ObjectMapper json; public OrderService(JdbcTemplate jdbc, ObjectMapper json) { this.jdbc = jdbc; this.json = json; } /** One database transaction. No Kafka call anywhere in the request path. */ @Transactional public UUID placeOrder(PlaceOrder cmd) { UUID orderId = UUID.randomUUID(); jdbc.update( "INSERT INTO orders (order_id, customer_id, amount_minor, currency, status) VALUES (?, ?, ?, ?, 'PLACED')", orderId, cmd.customerId(), cmd.amountMinor(), cmd.currency()); OrderPlaced event = new OrderPlaced(UUID.randomUUID(), orderId, cmd.customerId(), cmd.amountMinor(), cmd.currency(), Instant.now()); jdbc.update( "INSERT INTO outbox (event_id, topic, record_key, payload) VALUES (?, ?, ?, ?)", event.eventId(), "orders.order-placed", orderId.toString(), toJson(event)); return orderId; } private String toJson(OrderPlaced event) { try { return json.writeValueAsString(event); } catch (JsonProcessingException e) { throw new IllegalStateException("Cannot serialise " + event, e); } }}Why it is written this way
- Both inserts are in one database transaction, so the order and the fact that an event is owed commit or roll back together. This is the entire trick.
- The request no longer depends on Kafka being reachable. A broker outage delays events; it does not reject orders.
- The payload is serialised here, at the moment the fact is created, so the relay never has to rebuild an event from tables that may have changed since.
package com.example.orders;import java.util.List;import java.util.concurrent.ExecutionException;import java.util.concurrent.TimeUnit;import java.util.concurrent.TimeoutException;import org.springframework.jdbc.core.JdbcTemplate;import org.springframework.kafka.core.KafkaTemplate;import org.springframework.scheduling.annotation.Scheduled;import org.springframework.stereotype.Component;import org.springframework.transaction.PlatformTransactionManager;import org.springframework.transaction.support.TransactionTemplate;@Componentpublic class OutboxRelay { private record OutboxRow(long id, String topic, String key, String payload) {} private final JdbcTemplate jdbc; private final KafkaTemplate<String, String> kafka; private final TransactionTemplate tx; public OutboxRelay(JdbcTemplate jdbc, KafkaTemplate<String, String> kafka, PlatformTransactionManager transactionManager) { this.jdbc = jdbc; this.kafka = kafka; this.tx = new TransactionTemplate(transactionManager); } @Scheduled(fixedDelay = 500) public void publishPending() { tx.executeWithoutResult(status -> { List<OutboxRow> rows = jdbc.query(""" SELECT id, topic, record_key, payload FROM outbox WHERE published_at IS NULL ORDER BY id LIMIT 100 FOR UPDATE SKIP LOCKED """, (rs, n) -> new OutboxRow(rs.getLong("id"), rs.getString("topic"), rs.getString("record_key"), rs.getString("payload"))); for (OutboxRow row : rows) { publish(row); jdbc.update("UPDATE outbox SET published_at = now() WHERE id = ?", row.id()); } }); } /** Blocks until the broker has acknowledged the record, or fails the whole batch. */ private void publish(OutboxRow row) { try { kafka.send(row.topic(), row.key(), row.payload()).get(10, TimeUnit.SECONDS); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new IllegalStateException("Interrupted while publishing outbox row " + row.id(), e); } catch (ExecutionException | TimeoutException e) { throw new IllegalStateException("Publishing outbox row " + row.id() + " failed, will retry", e); } }}Why it is written this way
- The relay is itself a dual write (send, then mark as published), but a safe one: a crash between the two leaves the row unpublished, so it is sent again. The failure mode is a duplicate, never a loss.
- That is why the relay is at-least-once and every consumer of this topic must be idempotent on
event_id. Producer idempotence does not help: a secondsendof the same row is a new record to the broker. - A failed send throws, the database transaction rolls back and all rows of the batch stay unpublished. Rows already sent in this batch will be sent again.
FOR UPDATE SKIP LOCKEDlets several relay instances run without publishing the same row at the same moment, at the price of per-key ordering between instances. Run one active relay, or see the outbox lesson for ordered alternatives.@Scheduledneeds@EnableSchedulingon a configuration class. This service sets notransaction-id-prefix, so the only transaction manager is the database one.
Configuration
Settings that decide where the boundary sits. Kafka defaults are from the 4.1 configuration reference; Spring behaviour is from the Spring for Apache Kafka 3.3 and Spring Boot 3.5 references.
spring.kafka.producer.transaction-id-prefixDefault none
- What it does
- Makes the producer factory transactional. Spring Boot then also configures a
KafkaTransactionManagerand wires it into the listener container. The prefix must differ on each application instance. - When to change it
- Set it for read-process-write between topics. Setting it does not bring a database into the transaction; it only lets Spring order the two commits.
isolation.level (consumer)Default read_uncommitted
- What it does
- With
read_committed,pollreturns transactional records only once their transaction has committed, and nothing beyond the first still-open transaction. - When to change it
- Set
read_committedon every consumer of a topic written transactionally. With the default, consumers also receive records from aborted transactions, which defeats the rollback.
transaction.timeout.ms (producer)Default 60000 (1 minute)
- What it does
- How long a transaction may stay open before the coordinator aborts it.
- When to change it
- A slow database call inside a synchronised transaction counts against it. Keep database work short rather than raising this; a long open transaction also holds back
read_committedconsumers.
enable.auto.commit (consumer)Default true (Spring for Apache Kafka sets it to false unless you set it explicitly)
- What it does
- Whether the consumer commits offsets on a timer, independently of whether processing finished.
- When to change it
- Leave it off, as Spring does, so that offsets advance only after your database write has committed. Timed commits can move the offset past a record whose write later fails.
transaction.two.phase.commit.enable (producer)Default false (listed from the 4.1 reference; absent in 4.0)
- What it does
- From KIP-939: tells the broker that this producer takes part in a two-phase commit driven by an external coordinator, so its transactions do not time out.
- When to change it
- Leave it alone unless a framework that implements such a coordinator tells you to set it. It does not make Spring's
@Transactionalatomic across Kafka and JDBC.
Hands-on exercise
Find the edge of the transaction
Use the delivery simulator in transactional mode to see which effects a Kafka transaction rolls back after a crash, and which effect it cannot reach.
Steps · 0 of 6 done
Transactional mode with a crash before commit
Simulation · not a real clusterThe Kafka transaction aborts cleanly. Watch what that does and does not undo for the payment.
What you should see
After the crash the output record is aborted and the offset has not moved, so Kafka's own state shows no trace of the attempt. The payment side effect did happen, and it happens again on redelivery unless the idempotent consumer is on. Only the crash after the commit is free of repeats.
What can go wrong
Trusting @Transactional to cover the send
- Symptom
- A few orders per month are in the database with status PLACED and no downstream service has ever heard of them. They cluster around deploys.
- Cause
- The method's transaction is the database's. The Kafka send is a second, independent write; the process stopped, or the send failed, after the commit.
- Fix
- Write the event to an outbox table in the same database transaction and publish from there. Add a query that alerts on old unpublished rows.
Phantom events from a rollback
- Symptom
- The payment service logs charges for order ids that the order service cannot find.
- Cause
- The event was sent inside the method, and the database then rolled back (constraint violation, deadlock, lost connection). The record stays in the partition.
- Fix
- Never send from inside an open database transaction. Use the outbox, so that an event can only exist if its row committed.
Reading uncommitted records downstream
- Symptom
- A consumer acts on events that the producing service aborted, even though that service uses Kafka transactions correctly.
- Cause
- The consumer runs with the default
isolation.level=read_uncommitted, which returns aborted transactional records as well. - Fix
- Set
isolation.level=read_committedon every consumer of a transactionally written topic.
A non-idempotent write under a synchronised listener
- Symptom
- A ledger shows the same payment credited twice after a broker restart, although the service uses exactly-once settings.
- Cause
- The database committed, the Kafka commit that follows it failed, and the record was redelivered. The second run inserted a second row.
- Fix
- Make the database write idempotent: a unique constraint on the business key, or a processed-events row in the same database transaction.
Holding the Kafka transaction open across slow work
- Symptom
- Intermittent aborted transactions and producer errors during database slowness;
read_committedconsumers of the output topic stall. - Cause
- Slow database or HTTP work inside the transaction exceeds
transaction.timeout.ms, so the coordinator aborts it. - Fix
- Keep remote calls out of the transaction, shorten the database work, and prefer an outbox over a synchronised Kafka transaction for database-backed services.
How to troubleshoot
Orders exist but no event was published
Check
SELECT o.order_id FROM orders o LEFT JOIN outbox x ON x.record_key = o.order_id::text WHERE x.id IS NULL AND o.created_at < now() - interval '5 minutes';
- How to read it
- Rows returned were written without an outbox row: some code path still does a dual write. With an outbox in place this query must always be empty.
Events are delayed by minutes
Check
SELECT count(*), min(created_at) FROM outbox WHERE published_at IS NULL;
- How to read it
- A growing count with an old minimum means the relay is stopped or failing to publish. Look for its send errors; orders are safe but downstream services are waiting.
A consumer seems stuck although the producer is writing
Check
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group payment-service
- How to read it
- Lag that grows on one partition while
CURRENT-OFFSETstands still can be aread_committedconsumer waiting behind an open transaction. Find the producer that started a transaction and never finished it.
Downstream services saw an event the producer says it rolled back
Check
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic payments.payment-result --from-beginning --isolation-level read_committed
- How to read it
- If the record is absent here but your service received it, the service is reading uncommitted. If it is present, the Kafka transaction did commit and the rollback was on the database side only.
Key takeaways
- A database commit and a Kafka send are two commits in two systems. Whatever order you pick, a crash between them leaves one side without the other.
- A Kafka transaction is atomic over Kafka partitions and consumer offsets, and over nothing else.
- Spring's transaction synchronisation commits one transaction and then the other. It narrows the window and chooses its direction; it does not close it.
- On the producer side use an outbox or CDC: one database transaction, with publishing derived from committed rows.
- On the consumer side rely on redelivery and make the database write idempotent, or store the offset in the same database transaction as the data.
- Every one of these designs delivers at least once somewhere, so downstream consumers still need to deduplicate.
Practical check
Small scenarios, one decision each. Pick what you would do, then read why.
Scenario 1 of 3
An order service runs @Transactional placeOrder(): it inserts the order, then calls kafkaTemplate.send. No transaction-id-prefix is set. After a rolling deploy, support reports 3 orders from that minute with status PLACED and no payment attempt. The topic has no events for them and the broker logs show no errors.
What should the team change?
Scenario 2 of 3
A payment listener runs in a container-managed Kafka transaction and its method is @Transactional on the JDBC manager. It inserts into ledger_entries (no unique constraint) and sends payment-result. During a broker restart, 14 ledger entries appear twice, while payment-result contains each result exactly once.
Which explanation fits both observations?
Scenario 3 of 3
A consumer builds an account_balance table from ledger.entries (6 partitions). Each event adds an amount to a balance, so applying one twice is wrong, and the events carry no unique id. The service writes to no other topic. The team wants each event applied exactly once in the table.
Which design achieves that with the least machinery?
References
References: Apache Kafka documentation: Design, message delivery semantics and using transactions ↗ · KIP-98: Exactly Once Delivery and Transactional Messaging ↗ · KafkaProducer Javadoc (4.1): idempotent and transactional producer ↗ · Spring for Apache Kafka 3.3 reference: Transactions ↗ · Spring for Apache Kafka 3.3 reference: Exactly Once Semantics ↗ · Spring for Apache Kafka 3.3 reference: Examples of Kafka transactions with other transaction managers ↗ · Apache Kafka 4.1 producer configuration reference ↗ · KIP-939: Support Participation in 2PC ↗ · Report a problem