Skip to content

Path 7 · lesson 2 of 8

Advanced35 minDebezium 3.7 (the 'stable' documentation at the time of writing) · Apache Kafka 4.x with Kafka Connect (docs read for 4.1) · PostgreSQL with pgoutput · Spring for Apache Kafka 3.3 · Java 21

Change data capture with Debezium

Log-based CDC reads the database's write-ahead log and turns every committed row change into a Kafka record. How Debezium does it on Kafka Connect, and what to watch in production.

What you will learn

  • Explain how log-based CDC differs from polling a table, and why it sees every committed change in commit order
  • Describe how a Debezium PostgreSQL connector runs inside Kafka Connect: replication slot, snapshot, streaming, stored offsets
  • Read a Debezium change event (before, after, source, op) and handle deletes and tombstones in a consumer
  • Configure the outbox event router so an outbox table becomes clean business events on a topic per aggregate type
  • Recognise and prevent the main production risks: WAL retained by an idle slot, schema changes, duplicates after a restart
  • Choose between a polling publisher and CDC for a given service

Before you start

The concept

Every relational database already keeps an ordered record of each committed change: PostgreSQL's write-ahead log (WAL), MySQL's binlog. The database writes it for crash recovery and replication. Log-based change data capture (CDC) reads that log as a client and emits one event per inserted, updated or deleted row. The application does nothing extra: if a row changed and the transaction committed, the change is in the log.

Debezium is an open-source set of CDC connectors. Most commonly it runs as Kafka Connect source connectors: Kafka Connect is the framework, shipped with Apache Kafka, that runs connectors in worker processes, stores their configuration and progress in Kafka topics, and restarts their tasks on another worker when one fails. Debezium is not part of Apache Kafka. It is a separate project that you add to a Connect cluster as a plug-in.

  • The PostgreSQL connector reads a logical replication stream through a replication slot, using the pgoutput plug-in that PostgreSQL itself uses for logical replication.
  • By default each captured table gets its own topic, named topicPrefix.schemaName.tableName, for example shop.public.orders.
  • Each record's key is the row's primary key, and its value is an envelope with the row state before and after the change, the operation op, and source metadata such as the transaction id and log position.
  • Connect records how far the connector has read (its offsets) in a Kafka topic, so a restarted connector continues where it stopped.

For the order system, CDC has two uses. It can be the relay for the outbox: Debezium captures inserts into outbox_event, and its outbox event router turns each one into a business event on orders.events. It can also publish raw row changes, for example every change to the stock_level table, for consumers that maintain a cache, a search index or a reporting copy.

Polling publisher

  • A scheduled query in your own service: nothing new to deploy
  • Works on any database with transactions
  • Latency is the poll interval, and each poll costs a query even when idle
  • Needs a published flag, an index and a cleanup job
  • You implement ordering, single-active relay and retries yourself
  • Cannot see deletes or intermediate updates of ordinary tables

Log-based CDC with Debezium

  • Needs a Kafka Connect cluster, the connector and database replication privileges
  • Needs a supported database with logical decoding or binlog access enabled
  • Low latency, with no queries against the tables while streaming
  • Outbox rows can be deleted in the inserting transaction: the log still has the insert
  • Changes arrive in commit order, read by a single task
  • A stopped connector makes the database retain WAL: a new operational risk

Why it exists

Before log-based CDC, there were three ways to get data out of a database and into other systems, and each loses something.

ApproachHow it worksWhat it misses
Dual write from the applicationThe service writes to the database and also to KafkaAtomicity: one of the writes can fail alone (see the outbox lesson). Changes made by scripts, migrations or other services are never published.
Polling with a timestamp or version columnSELECT ... WHERE updated_at > :last on a scheduleDeletes (the row is gone), intermediate states between two polls, and rows whose transaction committed late with an older timestamp. It also loads the primary with queries.
Triggers writing to an audit tableA trigger copies each change to another table, which is then polledNothing in principle, but every write pays for the trigger inside its transaction, and the audit table still has to be polled and cleaned.

The transaction log has none of these gaps. It contains every committed change, including deletes, in the order the database applied them, whoever made the change. Reading it adds no statements to the application's transactions. The cost is a dependency on database internals: log retention, replication privileges and a format that differs per database. Debezium's job is to hide that behind one event format.

For the outbox specifically, the polling relay in the previous lesson is correct but has limits: latency tied to the poll interval, a query load that grows with the poll rate, and code you own for ordering and failover. Replacing it with CDC removes the flag, the index, the lock and the polling loop. The Order Service keeps only the insert into outbox_event.

How it works inside

Slot, snapshot, streaming

  1. Prerequisites. PostgreSQL runs with wal_level=logical. The connector's database user has the REPLICATION privilege, and with pgoutput a publication lists the tables whose changes are published.
  2. Replication slot. On first start the connector creates a logical replication slot (default name debezium). A slot is the server's memory of one consumer: PostgreSQL keeps every WAL segment that the slot's consumer has not yet confirmed.
  3. Snapshot. PostgreSQL normally removes old WAL, so the log does not hold the full history. With the default snapshot.mode=initial, a connector with no stored offsets first reads the current contents of the captured tables in a consistent snapshot and emits them as events with op set to r (read).
  4. Streaming. The connector then streams changes from the exact log position of the snapshot, so nothing committed during the snapshot is missed. Each change becomes a record sent to Kafka through Connect's producer.
  5. Offsets. Connect periodically writes the connector's position (the log sequence number, LSN) to its offsets topic. After Kafka has the records, the connector confirms the position to PostgreSQL, which can then discard older WAL.

The PostgreSQL connector always uses a single task, whatever tasks.max says: one reader of one ordered log. That is what preserves commit order, and it also means throughput is bounded by one stream.

The change event, in outline

Record on shop.public.stock_level (JSON converter, schemas disabled)JSON
{  "before": { "sku": "SKU-42", "available": 7 },  "after":  { "sku": "SKU-42", "available": 6 },  "source": {    "connector": "postgresql",    "db": "inventorydb",    "schema": "public",    "table": "stock_level",    "txId": 584,    "lsn": 24064704,    "snapshot": false  },  "op": "u",  "ts_ms": 1556890294484}

Why it is written this way

  • op is c (create), u (update), d (delete), r (read during a snapshot) or t (truncate). The record key is the primary key, here the sku.
  • This example assumes REPLICA IDENTITY FULL on the table, which puts the whole old row in before at the cost of more WAL. With PostgreSQL's default setting only previous values of primary key columns are available, and for an update only if the key itself changed, so do not build logic on before without checking the setting.
  • source.lsn and source.txId identify the change in the log. A consumer can store the last LSN it applied to detect duplicates.
  • A delete is an event with op d and after null. By default (tombstones.on.delete=true) it is followed by a second record with the same key and a null value, so a compacted topic can drop the key.

The outbox event router

Consumers of business events should not have to unpack a row-change envelope of an outbox table. The outbox event router is a single message transformation (SMT) that runs inside the connector and rewrites each captured insert:

Outbox column (default name)Becomes
aggregatetypePart of the topic name. The default pattern is outbox.event. followed by the value. This lesson changes it to the value followed by .events
aggregateidThe record key, so all events of one aggregate share a partition
idA header named id: the event id consumers deduplicate on
payloadThe record value, and nothing else unless you add fields
type and other columnsNot emitted by default. table.fields.additional.placement can add them as headers or envelope fields

The router expects an outbox table to behave as a queue: inserts only. It drops delete events for the table automatically and, by default, logs a warning for updates. That is why the Order Service can insert the outbox row and delete it in the same transaction: the log contains both, and only the insert becomes an event.

What happens when something fails

Crash pointResulting stateRecovery
Order Service before its commitNothing in the WAL that will be decoded as committedNo event. The transaction never committed, so the connector has no change to publish for it.
Connect worker crashes after sending records, before the offset flushRecords are in Kafka, stored offset is olderAnother worker restarts the task from the stored offset and emits those changes again. Consumers deduplicate.
Connect worker stopped gracefullyOffsets recorded, slot remainsThe task moves to another worker and continues where it stopped.
Connector stopped or failed for hoursThe slot holds its position. PostgreSQL keeps all WAL since thenOn restart the connector catches up. If the disk fills first, the database stops accepting writes.
Kafka unavailableConnect cannot produce. The slot position does not advanceThe connector retries and resumes when brokers return. WAL accumulates meanwhile.
Slot dropped, or invalidated by a WAL size limitThe stored offset points at WAL that no longer existsChanges in the gap are lost to the stream. A new snapshot is required to resynchronise.
Database failover to a standbyThe slot may not exist on the new primaryFollow the connector's documented failover steps: recreate the slot before writes resume, and re-snapshot if in doubt.
Crash at each point, what is left behind, and how the pipeline recovers.

Ordering

The connector reads changes in commit order, and all changes to one row carry the same key, so they land in one partition in that order. Across different keys, and across different tables (which are different topics), Kafka gives no relative order. A transaction that touched three tables arrives as independent records on three topics. Consumers that need the whole transaction should be given one outbox event instead of being asked to reassemble it.

Architecture

Drawing diagram…

Flowchart: the Order Service writes orders and outbox rows in one PostgreSQL transaction. Both tables feed the write-ahead log, which a replication slot exposes as a logical replication stream to the Debezium connector running in a Kafka Connect worker. The outbox event router transformation produces records to the orders.events topic, the connector stores its log position in the connect-offsets topic and confirms it back to the slot. Payment and Inventory consume the topic.

flowchart LR
  OS["Order Service"] -->|"one transaction"| PG
  subgraph PG["PostgreSQL, order database"]
    OT["orders"]
    OB["outbox_event"]
    WAL["Write-ahead log"]
    SLOT["Replication slot order_outbox"]
  end
  OT -.-> WAL
  OB -.-> WAL
  WAL --> SLOT
  subgraph KC["Kafka Connect worker"]
    DBZ["Debezium PostgreSQL connector"]
    SMT["Outbox event router"]
  end
  SLOT -->|"logical replication stream"| DBZ
  DBZ --> SMT
  SMT -->|"key orderId, header id"| T["Topic orders.events"]
  DBZ -->|"last LSN read"| OFF["Topic connect-offsets"]
  DBZ -.->|"confirm flushed LSN"| SLOT
  T --> PS["Payment Service"]
  T --> IS["Inventory Service"]
Debezium is a replication client of the database and a producer to Kafka. The slot remembers its position on the database side, the offsets topic on the Kafka side.
  • The application's only responsibility is the insert into outbox_event in its transaction. Everything from the WAL onwards is infrastructure.
  • The connector and the transformation run in the same Connect task. The transformation changes the topic, key, headers and value before the record is produced.
  • Progress is kept in two places: the offsets topic tells the connector where to resume, and the confirmed position on the slot tells PostgreSQL which WAL it may delete. A connector that is not running confirms nothing.
  • Consumers see an ordinary Kafka topic. They do not know or care that the producer was a CDC connector.

Message flow

Drawing diagram…

Sequence diagram: the Order Service inserts an order and an outbox row, deletes the outbox row and commits. PostgreSQL streams the decoded changes to the Debezium connector, whose router publishes the insert to orders.events. The worker crashes before flushing its offset, restarts from an older log position, publishes the same event again, stores the new offset and confirms it to PostgreSQL. The Inventory Service processes the first copy and skips the second.

sequenceDiagram
  participant O as Order Service
  participant P as PostgreSQL
  participant D as Debezium connector
  participant K as Kafka
  participant I as Inventory Service
  O->>P: BEGIN
  O->>P: INSERT orders row
  O->>P: INSERT outbox_event row e1
  O->>P: DELETE outbox_event row e1
  O->>P: COMMIT, changes written to WAL
  P-->>D: decoded insert and delete at LSN 100
  D->>D: router keeps the insert, drops the delete
  D->>K: orders.events key o-1001 header id e1
  K-->>D: ack
  Note over D: worker crashes before the offset flush
  D->>K: read connect-offsets after restart
  K-->>D: last stored LSN 90
  D->>P: start replication from LSN 90
  P-->>D: insert at LSN 100 again
  D->>K: orders.events key o-1001 header id e1
  D->>K: store offset LSN 100
  D->>P: confirm flushed LSN 100
  K->>I: e1, first copy, reserve stock
  K->>I: e1, second copy, skipped by event id
An outbox insert travels through the WAL to Kafka. A worker crash before the offset flush replays the change, and the consumer skips the second copy.
  1. The outbox row never needs to be visible in the table. Its insert is in the log, and that is what the connector reads.
  2. Offsets are flushed periodically (the Connect worker default is every 60 seconds), not per record. Everything sent since the last flush can be sent again after a crash.
  3. The connector confirms a position to PostgreSQL only after it is safely recorded. Until then the WAL from that position onwards is retained.

In a real system

A typical setup for the order system runs a Kafka Connect cluster of three workers in distributed mode. Two connectors are registered. order-outbox-connector captures only public.outbox_event in the order database and routes to orders.events (12 partitions, key = order id), read by the payment-service, inventory-service and notification-service groups. inventory-stock-connector captures public.stock_level in the inventory database and writes raw change events to inventory.public.stock_level (key = sku), read by a service that maintains the product page's availability cache. Each connector has its own replication slot.

Invalidate from the change log

Simulation, not a real cluster

The animated lesson shows the second use: row changes from the log driving cache invalidation, with the ordering and lag effects visible.

Step 1 · The pattern

A copy with a timer

Readers ask the cache first. When the TTL runs out the key is dropped, the next reader fetches the row from the database and stores it again. Nothing is writing, so every copy is correct.

  • current data
  • request
Open the full animated lesson

When to use it

  • As the outbox relay when polling is too slow, too heavy on the database, or you have many services and want one standard relay instead of a hand-written one in each.
  • To feed derived stores (cache, search index, warehouse) from an existing table without changing the application that writes it.
  • To capture changes made outside the application as well: batch jobs, migrations, manual fixes.
  • During a migration, to keep a new system's copy of the data in step with the old one.

When not to use it

  • Nobody will operate Kafka Connect and watch the replication slot. A neglected slot can fill the database's disk, which is worse than a late event.
  • The database is a managed service or version where logical decoding cannot be enabled, or where you cannot get replication privileges.
  • One service with modest volume already has a working polling relay. CDC adds infrastructure without changing the guarantee.
  • As the contract between teams, when it would expose table structure. Publish outbox events instead.

Java and Spring Boot

A Debezium 3.7 PostgreSQL connector that relays the order outbox, registered with Kafka Connect's REST API (POST /connectors), then two Spring consumers: one for the routed business events and one for raw row changes. Option names and defaults were checked against the Debezium stable documentation. Confirm them for the version you deploy.

order-outbox-connector.jsonJSON
{  "name": "order-outbox-connector",  "config": {    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",    "tasks.max": "1",    "database.hostname": "orders-db",    "database.port": "5432",    "database.user": "debezium",    "database.password": "${file:/opt/connect/secrets/orders-db.properties:password}",    "database.dbname": "orderdb",    "topic.prefix": "orders-cdc",    "plugin.name": "pgoutput",    "slot.name": "order_outbox",    "publication.name": "order_outbox_pub",    "publication.autocreate.mode": "filtered",    "table.include.list": "public.outbox_event",    "tombstones.on.delete": "false",    "heartbeat.interval.ms": "10000",    "transforms": "outbox",    "transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",    "transforms.outbox.route.topic.replacement": "${routedByValue}.events",    "transforms.outbox.table.fields.additional.placement": "type:header:eventType",    "transforms.outbox.table.expand.json.payload": "true",    "key.converter": "org.apache.kafka.connect.storage.StringConverter",    "value.converter": "org.apache.kafka.connect.json.JsonConverter",    "value.converter.schemas.enable": "false"  }}

Why it is written this way

  • plugin.name must be set: the documented default is decoderbufs, which needs an extra server plug-in. pgoutput is built into PostgreSQL.
  • slot.name must be unique per connector, and may contain only lower-case letters, digits and underscores. The default name debezium collides as soon as a second connector appears.
  • publication.autocreate.mode=filtered creates a publication for the included tables only. The default all_tables publishes every table in the database.
  • table.include.list names only the outbox table. The Debezium documentation says a connector that applies the outbox router should capture outbox tables only.
  • route.topic.replacement turns aggregatetype = 'orders' into the topic orders.events. type:header:eventType copies the type column into a header so consumers can route without parsing the value.
  • StringConverter writes the key as the plain order id. table.expand.json.payload with the JSON converter and schemas disabled writes the payload as a JSON object, not as an escaped string.
  • tombstones.on.delete=false: deleting an outbox row must not produce a null-value record for that aggregate id. The password uses a Connect config provider. Never put it inline.

Configuration

Connector options and defaults from the Debezium 3.7 PostgreSQL connector and outbox event router documentation, plus the Connect worker and PostgreSQL settings that decide how much is replayed and how much WAL is kept.

plugin.nameDefault decoderbufs
What it does
The logical decoding output plug-in the connector reads from. Supported values are decoderbufs and pgoutput.
When to change it
Set pgoutput unless you have installed decoderbufs on the server. It is always present in PostgreSQL, including on managed services.
slot.nameDefault debezium
What it does
The logical replication slot the connector creates and reads. The slot stores the connector's position on the database side.
When to change it
Give every connector its own name. Two connectors on one slot will steal each other's changes.
snapshot.modeDefault initial
What it does
When to snapshot. initial snapshots only when no offsets are stored. Other modes include always, initial_only, no_data and when_needed.
When to change it
Keep initial for tables whose existing rows consumers need. no_data skips the data snapshot, suitable only when everything of interest is still in the WAL.
publication.autocreate.modeDefault all_tables
What it does
Whether and how the connector creates the pgoutput publication. all_tables creates one for every table in the database.
When to change it
Use filtered so only the included tables are published, or create the publication yourself and manage it with your migrations.
heartbeat.interval.msDefault 0 (no heartbeats)
What it does
How often the connector emits a heartbeat record. Heartbeats let it record progress even when no captured table changes.
When to change it
Set it when captured tables change rarely while the rest of the database is busy. Without heartbeats the connector has nothing to confirm and WAL accumulates.
tombstones.on.deleteDefault true
What it does
Whether a delete event is followed by a record with the same key and a null value, so a compacted topic can remove the key.
When to change it
Keep true for table topics that are compacted. Set false for an outbox connector.
transforms.outbox.route.topic.replacementDefault outbox.event.${routedByValue}
What it does
The destination topic name. ${routedByValue} is replaced by the value of the routing column (aggregatetype by default).
When to change it
Change it to match your topic naming. Create the topics beforehand with the partition count and replication factor you want.
transforms.outbox.table.op.invalid.behaviorDefault warn
What it does
What the router does when it sees an UPDATE on the outbox table: warn, error (both continue) or fatal (stop the connector).
When to change it
Leave at warn and never update outbox rows. This is the reason the CDC version has no published_at column.
offset.flush.interval.ms (Connect worker)Default 60000 (1 minute)
What it does
How often Connect tries to commit source offsets for tasks. It bounds how much is replayed after a worker crash.
When to change it
Lower it to reduce duplicates after a crash, at the cost of more writes to the offsets topic. It never removes the need for idempotent consumers.
max_slot_wal_keep_size (PostgreSQL)Default -1 (unlimited)
What it does
The maximum WAL that replication slots may retain. Beyond it, PostgreSQL removes WAL the slot still needs and the slot can no longer be used.
When to change it
Setting a limit protects the database's disk at the price of the connector: an invalidated slot means a new snapshot. Decide which failure you prefer, and alert long before either.

Hands-on exercise

Trace an outbox row through the log, and price an outage

Follow one event from the transaction to the consumer, find where a duplicate comes from, and work out how long a stopped connector can be tolerated before the database is at risk.

Steps · 0 of 5 done

Take it further in the lab: Transactional outbox

What can go wrong

A stopped connector fills the database disk
Symptom
Disk usage on the PostgreSQL primary climbs steadily. pg_wal is tens of gigabytes. Eventually writes fail and checkout is down.
Cause
A replication slot makes PostgreSQL keep all WAL its consumer has not confirmed. The connector was failed, paused, or deleted without dropping its slot, and with max_slot_wal_keep_size at its default there is no limit.
Fix
Alert on slot inactivity and retained WAL size. Restart or fix the connector. If a connector is retired, drop its slot. Consider a WAL size limit, knowing that hitting it invalidates the slot.
WAL grows although the connector is running
Symptom
The connector is healthy and lag metrics look fine, yet retained WAL keeps growing.
Cause
The captured tables change rarely while other tables or other databases on the same server are busy. The connector emits nothing, so no offset is committed and no position is confirmed to the server.
Fix
Set heartbeat.interval.ms. For the multi-database case, follow the connector documentation's guidance on a heartbeat table that produces a real change in the captured database.
Duplicates after a Connect restart
Symptom
After a worker crash or a rebalance of the Connect cluster, consumers see a burst of events they have already processed.
Cause
Offsets are flushed periodically. The task restarts from the last stored offset and emits everything after it again.
Fix
Deduplicate in every consumer: by the id header for outbox events, by source.lsn or an idempotent upsert for raw changes.
Schema changes surprise the consumers
Symptom
After a migration, a consumer of a table topic fails to deserialise, or silently reads null for a renamed column.
Cause
The envelope's before and after mirror the table's columns. PostgreSQL logical decoding does not carry DDL, so there is no schema change event to react to: the rows simply start arriving in the new shape.
Fix
Treat captured tables as a published schema: add columns as nullable, never rename or drop in one step, and use a schema registry with compatibility checks for table topics. Prefer outbox events for contracts between teams.
Updating or reusing outbox rows
Symptom
The connector log fills with warnings from the event router, and some expected events never appear.
Cause
The application updates outbox rows (for example a leftover published_at flag). The router treats the outbox as insert-only and does not emit events for updates.
Fix
Insert only. Remove status columns from the CDC version of the table and delete rows instead of updating them.
Snapshot of a large table at the wrong moment
Symptom
Registering a connector, or losing its offsets, floods a topic with millions of r events and puts read load on the primary for hours.
Cause
With snapshot.mode=initial a connector that finds no stored offsets snapshots every captured table. Renaming the connector or changing topic.prefix makes it look new.
Fix
Plan the first snapshot. Keep connector names and prefixes stable, restrict table.include.list, and read the documentation on incremental snapshots for large tables.

How to troubleshoot

No new events on the topic

Check

curl -s http://connect:8083/connectors/order-outbox-connector/status
How to read it
Kafka Connect's REST API reports the connector and task state. FAILED on the task comes with a stack trace in the trace field: fix the cause, then POST /connectors/order-outbox-connector/tasks/0/restart. A failed task does not restart by itself.
Database disk usage is climbing

Check

SELECT slot_name, active, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_wal FROM pg_replication_slots;
How to read it
active = false means no client is connected to the slot: the connector is down. A large retained_wal with active = true means the connector is connected but not confirming progress, which points to missing heartbeats or a connector that cannot produce to Kafka.
Events arrive late

Check

JMX metric `MilliSecondsBehindSource` on the MBean `debezium.postgres:type=connector-metrics,context=streaming,server=<topic.prefix>`
How to read it
It is the delay between a change in the database and the connector processing it. A steadily rising value means the single task cannot keep up or Kafka is slow to accept records. A flat low value means the delay is downstream, in the consumers.
Unsure where a connector will resume

Check

curl -s http://connect:8083/connectors/order-outbox-connector/offsets
How to read it
Shows the stored source offset, including the LSN. Compare it with confirmed_flush_lsn on the slot. If the stored LSN is no longer available on the server, the connector cannot continue from it and needs a snapshot.
Consumers lag behind the CDC topic

Check

kafka-consumer-groups.sh --bootstrap-server broker:9092 --describe --group inventory-service
How to read it
Lag here is an ordinary consumer problem, unrelated to the connector. After a snapshot or a catch-up the topic receives a burst, and lag that then shrinks steadily is normal.

Key takeaways

  • Log-based CDC reads the database's own transaction log: every committed change, including deletes, in commit order, with no extra statements in application transactions.
  • Debezium runs as Kafka Connect source connectors. It is a separate open-source project, not part of Apache Kafka.
  • A PostgreSQL connector snapshots first, then streams from a replication slot. Its position is stored in Connect's offsets topic and confirmed back to the slot.
  • Delivery is at-least-once: a restart replays everything since the last offset flush, so consumers deduplicate.
  • The outbox event router turns outbox inserts into business events: aggregatetype to topic, aggregateid to key, id to a header, payload to the value.
  • A slot nobody reads keeps WAL forever. Monitor slot activity and retained WAL before anything else.
  • Order is per key within a partition. Transactions across tables are not kept together.

Practical check

Small scenarios, one decision each. Pick what you would do, then read why.

Scenario 1 of 3

On Monday morning the order database's disk is at 94% and rising. pg_replication_slots shows slot order_outbox with active = false and 310 GB of retained WAL. The connector status shows its task FAILED since Saturday with an authentication error: the database password was rotated on Friday.

What should be done first?

Scenario 2 of 3

A Connect worker is killed by the node's out-of-memory killer. Within a minute the task is running on another worker. The Payment Service's dedupe table then rejects 1,150 OrderCreated events in a few seconds as already processed. The team is worried that the connector is broken.

What is the correct reading?

Scenario 3 of 3

A team of four runs one Spring Boot service with PostgreSQL and about 20 orders per minute. Events must reach other services within a few seconds. There is no Kafka Connect cluster in the company and nobody on call for the database at weekends. They are choosing a relay for their new outbox.

Which relay fits this situation?

References

References: Apache Kafka 4.1 documentation: Kafka Connect user guide (workers, REST API, offsets, exactly-once source support) ↗ · Apache Kafka 4.1: Kafka Connect configuration reference (offset.flush.interval.ms, header.converter) ↗ · Debezium documentation (stable, 3.7): Architecture ↗ · Debezium documentation (stable, 3.7): Connector for PostgreSQL (snapshots, change events, properties, WAL disk space, behaviour when things go wrong) ↗ · Debezium documentation (stable, 3.7): Outbox event router ↗ · Debezium blog: Reliable microservices data exchange with the outbox pattern ↗ · PostgreSQL documentation: replication settings (max_slot_wal_keep_size) ↗ · Spring for Apache Kafka 3.3 reference: @KafkaListener annotation ↗ · Chris Richardson, microservices.io: Transaction log tailing pattern ↗ · Report a problem