Source code of the Apache Airflow Tutorial for Beginners on YouTube Channel Coder2j (https://www.youtube.com/c/coder2j)

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Sync DAG changes from Git to Airflow

An open-source project dedicated to constructing robust data pipelines and scalable software infrastructure. We leverage industry-standard tools favored by developers to enhance efficiency and reliability. Uniquely, these pipelines are field-tested on farms across Sumatra, Indonesia, ensuring real-world applicability and resilience.

Repo for building docker based airflow image. Containers support multiple features like writing logs to local or S3 folder and Initializing GCP while container booting. https://abhioncbr.github.io/docker-airflow/

My self-learning about Apache Airflow

Airflow Pipeline for Machine Learning

🎵 LyricWave - Your AI Music Composer 🎶 Compose Unique MP4 Songs Effortlessly! LyricWave uses AI to create personalized music by harmonizing lyrics with captivating melodies and synthetic vocals. Unleash your musical creativity today! 🚀🎶

This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.

Apache Airflow Guide

An End-to-End ETL data pipeline that leverages pyspark parallel processing to process about 25 million rows of data coming from a SaaS application using Apache Airflow as an orchestration tool and various data warehouse technologies and finally using Apache Superset to connect to DWH for generating BI dashboards for weekly reports

From data gathering to model deployment. Complete ML pipeline using Docker, Airflow and Python.

Docker for airflow with mysql as backend

Building Data Warehouse on BigQuery which takes flat file as the data sources with Airflow as the Orchestrator

Quick Docker Compose deployment for Apache Airflow version 2.x. Ideal for local development

In this project, you will be building a Twitter Scheduler using Apache Airflow on Docker.

An airflow deployment configuration with sane defaults

A starting point for a data stack using Python, Apache Airflow and Metabase.

Run Apache Airflow using Docker containers

Tools to streamline Jupyter Notebook Prototypes into robust Data Products