redhat/granite-3-3b-a800m-instruct
IBM's Granite 3.0 large language model (LLM), optimized for local large language model operations
298
This Docker image is pre-configured to run IBM's Granite 3.0 large language model (LLM), optimized for local large language model operations. The image includes all necessary dependencies, libraries, and runtime environments for seamless deployment and interaction with Granite 3.0.
Key components:
To run the image in CLI mode use the command docker run --ipc=host -it redhat/granite-3-3b-a800m-instruct
To run the image as an openAI compatible API server run docker run --network host --ipc=host -it redhat/granite-3-3b-a800m-instruct -s
docker pull redhat/granite-3-3b-a800m-instruct