Postgres CDC using Debezium Kafka Connect as events
- cd docker
- set the correct env var using .env.example
- docker compose up -d
- Kafka Cluster
- Postgres Database
| Topic Name | Topic Partition | Topic CleanUp Policy | Topic Retention Period |
|---|---|---|---|
dz.configs |
compact | delete | 7 days |
dz.offsets |
compact | delete | 7 days |
dz.status |
compact | delete | 7 days |
- Kafka Topics
- Database Replication Slots
| Table Name | EH Name | EH Partition | EH CleanUp Policy | EH Retention Period | DB Slot Name | DB Publication Name |
|---|---|---|---|---|---|---|
| .<table_name> | dz.<schema>.<table_name> |
5 | delete | 7 days | <table_name>_dbz |
<table_name>_connection_dbz |
- kafka connect config in
docker/.env.example - sample table connector config templates in
connector-config-template/table-connectors
- comment out lines in
java.securityandjava.configdue to issues with off-the-shelfdebezium/connect:2.3.0.Finalimage connecting with Azure Postgres Flexible Server- Implemented Solution: https://stackoverflow.com/a/75827811