Skip to content

PipelineAI: End-to-End ML and AI Platform for Real-time Spark and Tensorflow Data Pipelines

License

Notifications You must be signed in to change notification settings

mkhaas/pipeline

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PipelineIO

Built with Love and Scale by ex-Netflix, Databricks, and Tensorflow Engineers

Tensorflow Databricks Netflix

Videos

Resources

Documentation

Workshops

Sign Up Now!

Workshop Architecture

Workshop Tools

Quick Start

Local Laptop

Kubernetes Mini + PipelineIO Cluster for your Laptop

  • PipelineIO Local (Local Laptop)

Kubernetes Docker

Kubernetes + PipelineIO Cluster on any Cloud or Data Center

  • Pipeline on AWS

AWS

Google Cloud Platform

Azure

Global 24x7 Support for Everyone!

Reference Architecture

Reference Architecture

Other Features

100% Open Source

Jupyter ZeppelinZeppelin Spark Tensorflow

100% Open Source using Kubernetes + Docker

PipelineIO Kubernetes Kubernetes Docker

PMML + Native C++/Java Code Generation for Optimized CPU/GPU Performance

Generate and Optimize Spark ML Model Nvidia GPU Intel CPU

Deploy directly from Jupyter Notebook to Hybrid-Cloud Microservices

Deploy Spark ML Model to Production

NetflixOSS-based Microservices for High Scalability and Availability

NetflixOSS-based Microservices

Continuous Train/Deploy Spark ML and Tensorflow AI Pipelines to Production

Kafka Kinesis KINESIS

Community

Global Advanced Spark and Tensorflow Meetup

Meetup

Slideshare Slides

Slideshare

YouTube Videos

YouTube

Docker Images

Docker

About

PipelineAI: End-to-End ML and AI Platform for Real-time Spark and Tensorflow Data Pipelines

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • HTML 98.8%
  • Protocol Buffer 0.4%
  • Scala 0.2%
  • Python 0.1%
  • JavaScript 0.1%
  • Shell 0.1%
  • Other 0.3%