More Docker. Easy Access. New Streamlined Plans. Learn more.

redhat/granite-3-3b-a800m-instruct

Verified Publisher

By Red Hat, Inc.

Updated 3 months ago

IBM's Granite 3.0 large language model (LLM), optimized for local large language model operations

Image
Data Science
Machine Learning & AI
Gen AI
0

298

This Docker image is pre-configured to run IBM's Granite 3.0 large language model (LLM), optimized for local large language model operations. The image includes all necessary dependencies, libraries, and runtime environments for seamless deployment and interaction with Granite 3.0.

Key components:

  1. Base Image: Starts with a lightweight, base image: ubi9
  2. Model Runtime Environment: The image provides llama.cpp as the model serving runtime

To run the image in CLI mode use the command docker run --ipc=host -it redhat/granite-3-3b-a800m-instruct

To run the image as an openAI compatible API server run docker run --network host --ipc=host -it redhat/granite-3-3b-a800m-instruct -s

Docker Pull Command

docker pull redhat/granite-3-3b-a800m-instruct