cssnr/chat-server repository overview

⁠Chat Server

⁠

• Setup⁠
  • Configure⁠
• Client⁠
  • Endpoints⁠
  • VitePress Plugin⁠
• Development⁠
• Support⁠
• Contributing⁠

Proxy Chat Server designed to work with the VitePress Chat⁠ plugin.
Secure your API key while live-streaming responses to the client.

Works with Claude, Gemini, OpenAI, or any OpenAI Compatible Provider⁠.

💯 100% Free to use with Zen OpenCode or Gemini Free Tier!

To get started Setup⁠ and Configure⁠ the server. No API Key required!

• Client: https://github.com/cssnr/vitepress-chat⁠
• Server: https://github.com/cssnr/chat-server⁠

⁠Features

• Works with Claude, OpenAI, Gemini and OpenAI Compatible Providers
• Includes Chat, Completion, and Object Endpoints
• Supports Multiple Clients Simultaneously
• Live Streams the Results to the Client
• Automatic Input Token Caching
• Automatic Retry on API Failures
• Deploy with Docker or Node
• Plus all the Client Features⁠

Built with the AI SDK⁠.

⁠Setup

💡 The server works out-of-the-box with NO environment variables.

With Docker.

docker run -p 3000:3000 cssnr/chat-server:latest

With Docker Compose.

services:
  chat:
    image: cssnr/chat-server:latest
    environment:
      MODEL: 'gemini-2.5-flash'
      GOOGLE_GENERATIVE_AI_API_KEY: 'xxx'
    ports:
      - '3000:3000'

With Node.

npm i
npm start

Note: you will need to export your environment variables or set them in the settings.env file.

For a Docker Swarm + Traefik + Basic Auth example see the docker-compose-swarm.yaml⁠.

For a Portainer Deploy workflow see the .github/workflows/deploy.yaml⁠.

⁠Configure

💡 All variables are optional. The default big-pickle model works with NO API Key.

Environment Variables (can be placed in a settings.env file).

You must also set the API key for the MODEL you select.

The PROVIDER_API_KEY is optional for free-tier models like big-pickle.

⁠INSTRUCTIONS

There are mechanisms to override the instructions per-call for all clients on all endpoints.
These are used as fallback when those instructions are not sent for configurations where this is desired.

The INSTRUCTIONS variable (legacy) also points to the INSTRUCTIONS_CHAT variable (recommended).

To disable the clients ability to send custom instructions set DISABLE_CLIENT_INSTRUCTIONS=true

⁠PROVIDER_OPTIONS

Provider Options: https://vercel.com/docs/ai-gateway/models-and-providers/provider-options⁠

For example, to disable "Reasoning" on big-pickle model.

{ "zen": { "thinking": { "type": "disabled" } } }

You may need to wrap the variable in single quotes.

PROVIDER_OPTIONS='{"openai":{"serviceTier":"flex","reasoningEffort":"low"}}'

You are responsible for providing valid options for the chosen model. The SDK supports providing provider options for multiple providers simultaneously. The value is only checked for valid JSON at startup and will fail at runtime if it contains invalid options.

⁠PROVIDER_USER_AGENT

If BASE_URL, MODEL, and PROVIDER_API_KEY are not set (Default Zen), a custom header is set:

User-Agent: opencode/version

This header increases the rate limit for request for the free OpenCode Zen endpoint.

To disable this without changing one of the above variables, you can set an empty value:

PROVIDER_USER_AGENT=

Otherwise, you can set a custom User-Agent (prefix) to anything you choose:

PROVIDER_USER_AGENT='my-app/1.0'

NOTE: The AI SDK appends a suffix to the UA: <userAgent> ai-sdk/provider-utils/x runtime/node

⁠Client

⁠Endpoints

Note: The / endpoint (legacy) also points to the /chat endpoint (recommended).

⁠chat

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat⁠

import { useChat } from '@ai-sdk/vue'
import { DefaultChatTransport } from 'ai'

const { messages, sendMessage, status, stop } = useChat({
  transport: new DefaultChatTransport({
    api: 'https://chat-server.cssnr.com/chat',
    headers: { Authorization: 'Basic Abc123=' },
    body: { instructions: 'You are a helpful assistant.' },
  }),
})

To send System Instructions from the client, add them to the body.

⁠completion

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-completion⁠

import { useCompletion } from '@ai-sdk/vue'

const { completion, complete, isLoading, stop } = useCompletion({
  api: 'https://chat-server.cssnr.com/completion',
  headers: { Authorization: 'Basic Abc123=' },
  body: { instructions: 'You are a helpful assistant.' },
})

await complete('Explain how to set up cssnr/chat-server')

To send System Instructions from the client, add them to the body.

⁠object

Reference: https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-object⁠

import { useObject } from '@ai-sdk/vue'
import { z } from 'zod'
import { zodToJsonSchema } from 'zod-to-json-schema'

const schema = z.object({ name: z.string(), age: z.number() })

const { object, submit } = useObject({
  api: 'https://chat-server.cssnr.com/object',
  headers: { Authorization: 'Basic Abc123=' },
  schema,
})

submit({
  instructions: 'You are a helpful assistant.',
  prompt: 'Extract the name and age from: John is 30 years old.',
  output: zodToJsonSchema(schema),
})

To send System Instructions and Output Schema from the client, add them to the body.

Note: Both instructions and output are custom body parameters parsed by the server.

⁠VitePress Chat Plugin

The client is currently available as a VitePress Plugin.

• https://github.com/cssnr/vitepress-chat⁠

⁠Development

To enable debug logs set: DEBUG=app

This works with no configuration using the big-pickle model.
You can set your environment variables in the settings.env file.
If using big-pickle for testing it is much faster to disable reasoning.

PROVIDER_OPTIONS='{"zen":{"thinking":{"type":"disabled"}}}'

In all cases you can set the PORT environment variable.

With Node run.

npm run dev

Point your client to: http://localhost:3000/⁠

With Docker compose (you may need to touch settings.env).

docker compose -f docker-compose-dev.yaml up --watch --build --remove-orphans

Point your client to: http://localhost/⁠

⁠Building

To build and test the docker image run.

bash build.sh
docker compose up

⁠Support

If you run into any issues or need help getting started, please do one of the following:

• Report an Issue: https://github.com/cssnr/chat-server/issues⁠
• Q&A Discussion: https://github.com/cssnr/chat-server/discussions/categories/q-a⁠
• Request a Feature: https://github.com/cssnr/chat-server/issues/new?template=1-feature.yaml⁠
• Chat with us on Discord: https://discord.gg/wXy6m2X8wY⁠

⁠Contributing

Please consider making a donation to support the development of this project and additional⁠ open source projects.

For a full list of current projects visit: https://cssnr.github.io/⁠