Public Repository

Last pushed: 6 days ago
Short Description
PipelineIO: End-to-End, Real-time Platform for Spark ML and Tensorflow AI Model Training and Serving
Full Description

PipelineIO: End-to-End ML and AI Platform for Real-time Spark and Tensorflow Data Pipelines

Highlights:
1) Easily Train and Deploy your Spark ML and Tensorflow AI Pipelines from Local Notebooks to Production Servers
7) Rapid Model A/B and Multi-armed Bandit Testing in Production with Complete Versioning and Rollback Support
2) Supports Jupyter/iPython Notebook, Zeppelin, Spark, Tensorflow, HDFS, S3, Kubernetes, Docker
3) Runs on all Cloud Providers including AWS, Google, Azure - as well as On-Premise
4) Full GPU Support for both Training and Prediction Workloads
5) Deploy Models as Optimized PMML or Generated Java and C++ Code for Native Performance on CPU/GPU
6) Deploy to High Performance, Highly-Scalable, Highly-Available Netflix-based Microservices
7) Full Monitoring and Metrics for All Training and Deployment Pipelines using Airflow and Prometheus
8) Ingests Many Streaming Sources including Kafka, AWS Kinesis, Google Dataflow, and Apache Beam
9) Built with Love and Scale by ex-Netflix, Databricks, and Tensorflow Engineers
10) 24x7 Support for Everyone - All Issues Will Be Addressed!

Docker Pull Command
Owner
fluxcapacitor

Comments (2)
rjmyoffice
8 months ago

It was the Coolest Hands on Demo ever...Thanks Chris

jaychen
2 years ago

thanks, the talk will help me a lots.