axisecp/larod-inference-server repository overview

⁠Larod inference server

In order to make larod available also to clients that do not reside on the very same device, which is indeed the case in real machine learning applications (such as e.g. the Watchlist application⁠), we need to supply a network protocol-based interface. The standard way to do this is to provide an inference server—such as the OpenVINO™ model server⁠ or the NVIDIA TensorRT Inference Server⁠—and that is what is provided for larod here.

⁠API

The larod inference server currently serves the TensorFlow⁠ gRPC⁠ interface (just like the OpenVINO™ model server). This makes it interchangable with other TensorFlow-based inference servers, which is exactly what we want.

⁠Build

Example docker build command:

docker image build --build-arg PROXY=http://wwwproxy:3128 -t axisecp/larod-inference-server:2.0.0-armv7hf .
docker push axisecp/larod-inference-server:2.0.0-armv7hf