Before running the POC, ensure you have the following installed:
- Apache NiFi - For orchestrating data flows.
- PostgreSQL - As the primary database system.
- Debezium - For change data capture.
- Apache Kafka - For messaging and streaming data.
- ZooKeeper - Required for Kafka coordination.
- Kafka UI - Optional UI for monitoring Kafka topics.
- NiFi Registry - For version control and management of NiFi flows.
docker-compose up --build
Once all the applications are up and running within Docker Compose, an additional step is required to properly configure the system. Follow the steps below:
-
Ensure all applications are up by running Docker Compose.
-
Execute the CURL command as defined in the connector/request_config.txt file to perform the final system configuration.
| Application | Host |
|---|---|
| NIFI | http:https://localhost:8080/nifi |
| NIFI REGISTR | http:https://localhost:18080/nifi-registry |
| POSTGRESQL | http:https://localhost:5432 |
| DEBEZIUM | http:https://localhost:8083 |
| KAFKA | http:https://localhost:9092 |
| KAFKA UI | http:https://localhost:9090 |
| ZOOKEEPER | http:https://localhost:2181 |
If you'd like to contribute to this POC, feel free to fork the repository, make your changes, and submit a pull request. Contributions, suggestions, and improvements are welcome!
This POC is licensed under the MIT License.
This POC is provided as-is without any warranties. It's intended for demonstration and educational purposes only. Use it at your own risk.
For any questions or inquiries, please contact [maintainer email or username].