vamman/turbo-7b repository overview

⁠TurboPlay 7 Billion Large Language Model

⁠Overview

This repository contains the Docker configuration and resources for deploying and using TurboPlay's state-of-the-art 7-billion-parameter large language model. The model is designed to excel in a variety of natural language processing tasks including but not limited to text generation, summarization, translation, and more.

Check out the LIVE version: https://turbo.gg⁠

⁠Usage

⁠Docker Image

The language model is encapsulated in a Docker image for seamless deployment and integration. To pull the Docker image, use the following command:

docker pull turboplay/language-model:7billion

⁠Running the Container

docker run -d -p 8080:8080 vamman/turbo-7b:latest

This command starts the container in detached mode and maps port 8080 on the host to port 8080 in the container.

⁠API Endpoint

The language model exposes a RESTful API for easy interaction. The API endpoint for making predictions is:

import requests
data = {"prompt": "Hi there, how's it going?"}
res = requests.post("http://127.0.0.1:8080/v1/models/model:predict", json=data)
print(res.json())http://localhost:8080/predict

Send POST requests to this endpoint with the input text, and the model will respond with its predictions.

⁠Configuration

The Docker image comes pre-configured with optimal settings for most use cases. However, advanced users can customize the model behavior and settings by modifying the configuration files provided in the repository.

⁠Issues and Support

If you encounter any issues or have questions about using the language model, please open an issue in this repository. The TurboPlay team actively monitors and responds to community feedback.

⁠License

This model is licensed under the Apache 2.0 license. This model was originally based on Falcon-7B model.