We are currently trying to use Debezium for capturing changes of 4 tables in a Postgres database. We are currently aware that for this use case we could use a kafka-streams app to join/aggregate tables for a KTable, however we want to keep kafka-stream topology simple, so the idea would be to use a Materialized View from Postgres and capture it changes.
Is it possible to do this, if so than how should we configure it?
Kafka Connect Source Configuration:
{
"name": "campaign-db-source-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url": "http://schema-registry:8081",
"value.converter.schema.registry.url": "http://schema-registry:8081",
"tasks.max": "1",
"database.hostname": "campaign-db",
"database.port": "5432",
"database.user": "postgres",
"database.password": "postgres",
"database.dbname" : "campaigndb",
"database.server.name": "campaign-db"
}
}
This config currently can capture all changes from individual tables but not from materialized view. (There is no topic created)