Stars
LETSQL provides a unified interface to multi-engine data pipelines. It is focused on composability with first-class support for UDFs 🚗🛠.
A repository with my config files and setup instructions for my terminal prompt on the Mac.
A lightweight CLI tool for versioning data alongside source code and building data pipelines.
🦉 ML Experiments and Data Management with Git
The mission of Project Drawdown is to help the world reach “Drawdown”— the point in the future when levels of greenhouse gases in the atmosphere stop climbing and start to steadily decline, thereby…
Backend Code for Massenergize Portal. This provides the API to the backend database, and is shared by the various front-end portal projects.
Ansible examples using Vagrant to deploy to local VMs.
Programming MapReduce with Scalding
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Ferry lets you define, run, and deploy big data applications on AWS, OpenStack, and your local machine using Docker
[PROJECT IS NO LONGER MAINTAINED] Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization fo…
Dockerfiles and scripts for Spark and Shark Docker images
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
scikit-learn: machine learning in Python