Events repository and API for product scans, photo uploads, robotoff annotations etc.

This is a repo with links to everything you'd ever want to learn about data engineering

VS Code in the browser

The lazier way to manage everything docker

Python SQL Parser and Transpiler

App to easily query, script, and visualize data from every database, file, and API.

Dynamically generate Apache Airflow DAGs from YAML configuration files

Move fast from data science prototype to pipeline. Capture, analyze, and transform messy notebooks into data pipelines with just two lines of code.

Docker Apache Airflow

The Amazon ECS CLI enables users to run their applications on ECS/Fargate using the Docker Compose file format, quickly provision resources, push/pull images in ECR, and monitor running application…

Easily run your Compose application to the cloud with compose-cli

CLI tool to visualise CloudFormation/SAM/CDK stacks as visjs networks, draw.io or ascii-art diagrams.

The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.

Coarse-grained lineage and tracing for machine learning pipelines.

Data Engineering Practice Problems

A Git Style Guide

This repo highlights a small sample of professional data engineering work I have done.

Python API for Open Food Facts (using a DB dump)

A simple and lightweight SQL client desktop with cross database and platform support.

Learn the entire ETL process based on Spotify API data

Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources.

troposphere - Python library to create AWS CloudFormation descriptions

A collection of useful CloudFormation templates

A reference architecture for deploying containerized microservices with Amazon ECS and AWS CloudFormation (YAML)

A comprehensive reference for all topics related to Natural Language Processing

A Data Engineering & Machine Learning Knowledge Hub