chiefj/ai-service repository overview

└── HTTP Client (Frontend / API consumer)
    └── [POST /api/v1/chat or /chat/persistent]
        └── HTTP Router (src/api/rest/v1/router.go)
            └── Route Matching & Middleware Injection
                └── AI Handlers (src/api/rest/v1/handlers/ai_handlers.go)
                    ├── Dependency Injection Container (src/shared/di/providers.go)
                    │   └── Resolves and injects the Orchestrator Service
                    │
                    └── AI Orchestrator Service (src/features/ai_orchestrator/service.go)
                        ├── Port: CloudflareClient (src/features/ai_orchestrator/ports.go)
                        │   └── Adapter: CloudflareAdapter (src/infra/adapters/cloudflare_adapter.go)
                        │       └── REST Calls -> External Cloudflare AI Gateway/Workers API
                        │
                        └── Port: MemoryRepository (src/features/ai_orchestrator/ports.go)
                            └── Adapter: MemoryRepository (src/infra/adapters/memory_repo.go)
                                └── Thread-safe In-Memory Vector DB (Cosine Similarity retrieval)

└── Incoming Chat Request
    ├── Route: /api/v1/chat (Stateless Flow)
    │   ├── [Step 1] Parse payload (messages list)
    │   ├── [Step 2] Send messages directly to Cloudflare LLM
    │   └── [Step 3] Return AI assistant response to the client
    │
    └── Route: /api/v1/chat/persistent (Stateful Flow)
        ├── [Step 1] Parse payload (user_id, message text)
        ├── [Step 2] Generate embedding vector for user message via Cloudflare AI
        ├── [Step 3] Fetch user memory history from In-Memory Memory Repository
        ├── [Decision] Does user have past conversational memory?
        │   ├── YES (Retrieve Context)
        │   │   ├── [Sub-Step] Compute Cosine Similarity between user message vector and all past vectors
        │   │   ├── [Sub-Step] Filter and sort memories where similarity >= threshold (default: 0.7)
        │   │   ├── [Sub-Step] Select top K context-rich memories (default: 5)
        │   │   └── [Sub-Step] Prepend retrieved context to the prompt as system instructions
        │   └── NO (Skip Context Retrieval)
        │       └── [Sub-Step] Proceed with empty conversational history context
        │
        ├── [Step 4] Send complete prompt (context + current message) to Cloudflare LLM
        ├── [Step 5] Receive LLM AI response text
        ├── [Step 6] Generate embedding vector for assistant response text via Cloudflare AI
        ├── [Step 7] Atomically store both User and Assistant turns in the Memory Repository
        └── [Step 8] Return AI response + semantic context metadata to the client

.
├── Dockerfile
├── docker-compose.yaml
├── go.mod
├── go.sum
├── test_api.sh
├── test_cloudflare_real.sh
├── contracts/
│   ├── changelog.md
│   └── openapi/
│       └── v1.yaml
├── scripts/
│   └── deploy_docker.sh
└── src/
    ├── main.go
    ├── api/
    │   └── rest/
    │       └── v1/
    │           ├── router.go
    │           └── handlers/
    │               └── ai_handlers.go
    ├── features/
    │   └── ai_orchestrator/
    │       ├── index.go
    │       ├── ports.go
    │       ├── service.go
    │       └── types.go
    ├── infra/
    │   └── adapters/
    │       ├── cloudflare_adapter.go
    │       └── memory_repo.go
    └── shared/
        ├── di/
        │   └── providers.go
        └── tracing/
            └── otel.go

export PORT=8080
export CF_ACCOUNT_ID="your-cloudflare-account-id"
export CF_API_TOKEN="your-cloudflare-api-token"
export CF_ACCESS_JWT_ASSERTION="your-cloudflare-access-jwt-assertion"

go run src/main.go

docker build -t chiefj/ai-service:latest .

./scripts/deploy_docker.sh

docker run -d \
  --name ai-service-test \
  -p 8080:8080 \
  -e CF_ACCOUNT_ID="your-cloudflare-account-id" \
  -e CF_API_TOKEN="your-cloudflare-api-token" \
  -e CF_ACCESS_JWT_ASSERTION="your-cloudflare-access-jwt-assertion" \
  chiefj/ai-service:latest

curl -s -f http://localhost:8080/api/v1/models

[
  {
    "id": "31097538-a3ff-4e6e-bb56-ad0e1f428b61",
    "name": "@cf/meta/llama-3-8b-instruct-awq",
    "description": "Quantized (int4) generative text model with 8 billion parameters from Meta.",
    "provider": ""
  },
  {
    "id": "053d5ac0-861b-4d3b-8501-e58d00417ef8",
    "name": "@cf/google/gemma-3-12b-it",
    "description": "Gemma 3 models are well-suited for a variety of text generation and image understanding tasks...",
    "provider": ""
  },
  {
    "id": "01bc2fb0-4bca-4598-b985-d2584a3f46c0",
    "name": "@cf/baai/bge-large-en-v1.5",
    "description": "BAAI general embedding (Large) model that transforms any given text into a 1024-dimensional vector",
    "provider": ""
  }
]

curl -s -f -X POST http://localhost:8080/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "@cf/meta/llama-3-8b-instruct",
    "messages": [
      {"role": "user", "content": "Explain Hexagonal Architecture in Go in one sentence."}
    ]
  }'

{
  "response": "In Go, Hexagonal Architecture, also known as Ports and Adapters Architecture, is a design pattern where the application's business logic is encapsulated in a \"domain layer\" and interacts with external dependencies through \"ports\" and \"adapters\", allowing for loose coupling and testability."
}

curl -s -f -X POST http://localhost:8080/api/v1/chat/persistent \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "live_user_123",
    "message": "Hello Cloudflare! My favorite programming language is Go.",
    "model": "@cf/meta/llama-3-8b-instruct",
    "embedding_model": "@cf/baai/bge-small-en-v1.5"
  }'

{
  "response": "Hello there! I'm thrilled to hear that your favorite programming language is Go! As a cloud-based service, Cloudflare is heavily invested in the Go programming language. In fact, our entire web server is written in Go, and it's been a fantastic choice for us.\n\nWe've found that Go's concurrency features, performance, and simplicity make it an ideal language for building scalable and reliable systems. Our Go-based server has allowed us to handle massive traffic spikes and provide fast and secure connections to our customers.\n\nWhat do you love most about Go? Is it the concurrency model, the simplicity of the language, or something else entirely?",
  "history": [
    {
      "role": "user",
      "content": "Hello Cloudflare! My favorite programming language is Go."
    },
    {
      "role": "assistant",
      "content": "Hello there! I'm thrilled to hear that your favorite programming language is Go! As a cloud-based service, Cloudflare is heavily invested in the Go programming language..."
    }
  ]
}

curl -s -f -X POST http://localhost:8080/api/v1/chat/persistent \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "live_user_123",
    "message": "What is my favorite programming language?",
    "model": "@cf/meta/llama-3-8b-instruct",
    "embedding_model": "@cf/baai/bge-small-en-v1.5"
  }'

{
  "response": "You told me earlier that your favorite programming language is Go!",
  "history": [
    {
      "role": "user",
      "content": "Hello Cloudflare! My favorite programming language is Go."
    },
    {
      "role": "assistant",
      "content": "Hello there!..."
    },
    {
      "role": "user",
      "content": "What is my favorite programming language?"
    },
    {
      "role": "assistant",
      "content": "You told me earlier that your favorite programming language is Go!"
    }
  ]
}