gencore/llama-cpp-qwen3 repository overview

⁠Overview

This project provides a Dockerized environment for running the Qwen3 language model. This model was released with a range of model sizes. This particular project is interested in running this model on the command line as part of natrual language processing tasks. The smallest models are packaged to provide options for the user.

0.6B is the smallest and shows a surprising amount of capability for its small <1GB size. The 1.7B model is the next size up and appears to be able to handle moderate text processing abilities.

⁠Key Features

• Multiple Model Sizes: Supports both 0.6B and 1.7B parameter models.
• Custom System Prompts: Easily set a custom system prompt to guide the model's behavior.
• Toggle Thought Process: Choose to show or hide the model's thought process in the output.
• Input Statistics: View the word count of your input prompt.
• Flexible Input: Provide user prompts as command-line arguments or via STDIN.

⁠Usage

You can interact with the model using various command-line options passed to the docker run command after the image name:

Usage: docker run --rm -i gencore/llama-cpp-qwen3:<tag> [options] [USER_PROMPT]

Options:
  --sys <prompt>        Set the system prompt.
  --hide-thoughts       Hide thought process in output.
  --hide-input-stats    Prevent printing of input word count.
  --help                Display this help message.

You can provide the USER_PROMPT as a command-line argument or pipe it to the container via STDIN.

Example:

# Using a command-line argument
docker run --rm -i gencore/llama-cpp-qwen3:0.6B "Hello, world!"

# Using STDIN
echo "Hello, world!" | docker run --rm -i gencore/llama-cpp-qwen3:0.6B