Sign inSign up

aarnphm/model-server

By aarnphm

ā€¢Updated about 5 years ago

Testing new Docker build process.

Image
0

2.2K

aarnphm/model-server repository overview

ā Unified Model Serving Framework Tweet

BentoML is an open platform that simplifies ML model deployment and enables you to serve your models at production scale in minutes

šŸ‘‰ Pop into our Slack community!ā  We're happy to help with any issue you face or even just to meet you and hear what you're working on :)

pypi_status downloads actions_status documentation_status join_slack

ā Why BentoML

  • The easiest way to turn your ML models into production-ready API endpoints.
  • High performance model serving, all in Python.
  • Standardized model packaging and ML service definition to streamline deployment.
  • Support all major machine-learning training frameworksā .
  • Deploy and operate ML serving workload at scale on Kubernetes via Yataiā .

ā Getting Started

  • Quickstart guideā  will show you a simple example of using BentoML in action. In under 10 minutes, you'll be able to serve your ML model over an HTTP API endpoint, and build a docker image that is ready to be deployed in production.
  • Main conceptsā  will give a comprehensive tour of BentoML's components and introduce you to its philosophy. After reading, you will see what drives BentoML's design, and know what bento and runner stands for.
  • ML Frameworksā  lays out best practices and example usages by the ML framework used for training models.
  • Advanced Guidesā  showcases advanced features in BentoML, including GPU support, inference graph, monitoring, and customizing docker environment etc.
  • Check out other projects from the BentoML teamā :

ā BentoServer base images

There are three type of BentoServer docker base image:

Image TypeDescriptionSupported OSUsage
runtimecontains latest BentoML releases from PyPIdebian, ubi{7,8}, amazonlinux2, alpine3.14production ready
cudnnruntime + support for CUDA-enabled GPUdebian, ubi{7,8}production ready with GPU support
develnightly build from development branchdebian, ubi{7,8}for development use only
  • Note: currently there's no nightly devel image with GPU support.

The final docker image tags will have the following format:

<release_type>-<python_version>-<distros>-<suffix>
   ā”‚             ā”‚                ā”‚        ā”‚
   ā”‚             ā”‚                ā”‚        ā””ā”€> additional suffix, differentiate runtime and cudnn releases
   ā”‚             ā”‚                ā””ā”€> formatted <dist><dist_version>, e.g: ami2, debian, ubi7
   ā”‚             ā””ā”€> Supported Python version: python3.7 | python3.8 | python3.9
   ā””ā”€>  Release type: devel or official BentoML release (e.g: 1.0.0)

Example image tags:

  • bento-server:devel-python3.7-debian
  • bento-server:1.0.0-python3.8-ubi8-cudnn
  • bento-server:1.0.0-python3.7-ami2-runtime

ā Latest tags for bento-server 1.0.0a5

ā debian 11 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]
ā debian 10 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]
ā UBI 8 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]
ā UBI 7 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]
ā amazonlinux 2 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]
ā alpine 3.14 [arm32v6, ppc64le, s390x, amd64, i386, arm32v5, riscv64, arm64v8, arm32v7]

Tag summary

Content type

Image

Digest

Size

408.2 MB

Last updated

about 5 years ago

docker pull aarnphm/model-server:devel-python3.8-centos7