Sign inSign up

aarnphm/bento-server

By aarnphm

ā€¢Updated over 4 years ago

Image
0

7.1K

aarnphm/bento-server repository overview

ā Unified Model Serving Framework Tweet

BentoML is an open platform that simplifies ML model deployment and enables you to serve your models at production scale in minutes

šŸ‘‰ Pop into our Slack community!ā  We're happy to help with any issue you face or even just to meet you and hear what you're working on :)

pypi_status downloads actions_status documentation_status join_slack

ā Why BentoML

  • The easiest way to turn your ML models into production-ready API endpoints.
  • High performance model serving, all in Python.
  • Standardized model packaging and ML service definition to streamline deployment.
  • Support all major machine-learning training frameworksā .
  • Deploy and operate ML serving workload at scale on Kubernetes via Yataiā .

ā Getting Started

  • Quickstart guideā  will show you a simple example of using BentoML in action. In under 10 minutes, you'll be able to serve your ML model over an HTTP API endpoint, and build a docker image that is ready to be deployed in production.
  • Main conceptsā  will give a comprehensive tour of BentoML's components and introduce you to its philosophy. After reading, you will see what drives BentoML's design, and know what bento and runner stands for.
  • ML Frameworksā  lays out best practices and example usages by the ML framework used for training models.
  • Advanced Guidesā  showcases advanced features in BentoML, including GPU support, inference graph, monitoring, and customizing docker environment etc.
  • Check out other projects from the BentoML teamā :

ā BentoServer base images

There are three type of BentoServer docker base image:

Image TypeDescriptionSupported OSUsage
runtimecontains latest BentoML releases from PyPIdebian{11,10}, ubi8, amazonlinux2, alpine3.14production ready
cudnnruntime + support for CUDA-enabled GPUdebian{11,10}, ubi8production ready with GPU support
develnightly build from development branchdebian{11,10}, ubi8for development use only
condaruntime + conda + optional GPU supportsdebian{11,10},production ready
  • Note: currently there's no nightly devel image with GPU support.

The final docker image tags will have the following format:

<release_type>-<python_version>-<distros>-<suffix>-<?:conda>
   ā”‚             ā”‚                ā”‚        ā”‚
   ā”‚             ā”‚                ā”‚        ā””ā”€> additional suffix, differentiate runtime and cudnn releases
   ā”‚             ā”‚                ā””ā”€> formatted <dist><dist_version>, e.g: ami2, debian, ubi7
   ā”‚             ā””ā”€> Supported Python version: python3.7 | python3.8 | python3.9
   ā””ā”€>  Release type: devel or official BentoML release (e.g: 1.0.0)

Example image tags:

  • bento-server:devel-python3.7-debian
  • bento-server:1.0.0-python3.8-ubi8-cudnn
  • bento-server:1.0.0-python3.7-ami2-runtime

ā NOTICE: MISSING PYTHON VERSION ON UBI

Python 3.7 and 3.10 is missing. The reason being RedHat doesn't provide support for these Python version. If you need to use UBI and Python 3.7 make sure to contact the BentoML team for supports..

ā NOTICE: CONDA AVAILABILITY ONLY ON DEBIAN

From 1.0.0a7 onwards, BentoML will only provide conda supports with debian variants only.

We ran into a lot of trouble building BentoML to supports Python from 3.6 to 3.10 with conda environment on other distros than debian. In order to reduce complexity we will now only provides conda on Debian-based image. Conda will be available with all of BentoML image type, including runtime, devel, cudnn.

If you need to use conda on other distros contact the BentoML team for supports.

Example conda tags:

  • bento-server:1.0.0a7-python3.8-debian11-runtime-conda
  • bento-server:1.0.0a7-python3.8-debian11-cudnn-conda
  • bento-server:devel-python3.8-debian11-cudnn-conda

ā Latest tags for bento-server 1.0.0a6

ā debian 11 [ amd64, arm64v8, ppc64le ]
ā debian 10 [ amd64, arm64v8, ppc64le ]
ā UBI 8 [ amd64, arm64v8, ppc64le ]
ā amazonlinux 2 [ amd64, arm64v8 ]
ā alpine 3.14 [ amd64, arm64v8, ppc64le, s390x ]

Tag summary

Content type

Image

Digest

Size

95.9 MB

Last updated

over 4 years ago

docker pull aarnphm/bento-server:1.0.0a7-python3.9-alpine3.14-runtime