gawsoft/mlserver-openvino repository overview

⁠Overview

This package provides a MLServer runtime compatible with Openvino. This package has couple features:

1. If server detect that model file is onnx format script will auto convert to openvino format (xml, bin) with dynamic batch size for openvino.
2. Openvino dynamic batch size
3. Grpc Ready
4. V2 Inference Protocol
5. Models metrics

Github repo: https://github.com/gawsoftpl/mlserver-openvino⁠

Docker image prepare for: https://gawsoft.pl⁠