data_engineering_pipeline

This repo will cover creating data engineering pipeline, from data ingestion, ETL, to deployment. Tools: Python, Apache Airflow, Apache NiFi, Kibana, PostgreSQL, pgAdmin, ElasticSearch, and others.