alokdekadev/tokensage-app repository overview

⁠TokenSage - Smart Token Management for AI Workflows

TokenSage optimizes AI costs by using a hybrid approach: OpenAI API for intelligent summarization and local LLM for content expansion. This reduces token usage while maintaining quality output.

Note: This is currently a testing release with a single local model (LLaMA 3.2 1B). Full source code and multi-model support will be available on GitHub in future releases.

⁠Core Concept

Problem: Large queries consume expensive API tokens
Solution: Condense with OpenAI → Expand with local LLM

1. Condense Step: Use OpenAI API to create intelligent summaries (minimal tokens)
2. Expand Step: Use local LLaMA 3.2 to generate detailed responses (zero API cost)
3. Result: Significant token savings with maintained quality

⁠Quick Start

⁠Prerequisites:

• Docker and Docker Compose installed
• At least 4GB RAM available
• OpenAI API key (for condense step)

⁠Setup Instructions:

1. Create docker-compose.yml file:

version: '3.8'

services:
  ollama:
    image: ollama/ollama:latest
    container_name: tokensage-ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0
    restart: unless-stopped

  tokensage-app:
    image: alokdekadev/tokensage-app:latest
    container_name: tokensage-app
    ports:
      - "8501:8501"
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    restart: unless-stopped

volumes:
  ollama_data:

2. Start everything:

docker compose up -d

3. Pull the local model:

docker exec tokensage-ollama ollama pull llama3.2:1b

4. Configure API key and use the app at http://localhost:8501⁠

⁠What You Get:

• Token Optimization: Reduce OpenAI API costs by 60-80%
• Hybrid Pipeline: Best of both worlds - cloud intelligence + local processing
• Streamlit Interface: Easy configuration and usage
• Memory Optimized: Works with 2-4GB RAM
• Testing Environment: Evaluate the concept with LLaMA 3.2 1B

⁠Use Cases:

• Content Creation: Summarize research, expand into articles
• Document Processing: Condense large documents, expand key points
• Research Analysis: Intelligent summarization with detailed explanations
• Cost-Sensitive Applications: Reduce API costs while maintaining quality

⁠Architecture:

User Query → OpenAI API (Condense) → Local LLM (Expand) → Final Output
   ↓              ↓                        ↓               ↓
Large Text    Smart Summary          Detailed Content   Cost Savings

⁠Commands:

# Start services
docker compose up -d

# Stop services
docker compose down

# View logs
docker compose logs -f

# Check status
docker compose ps

⁠Requirements:

• RAM: 2GB minimum, 4GB+ recommended
• Disk: ~3GB (images + model)
• Ports: 8501 (Streamlit), 11434 (Ollama)
• API: OpenAI API key for condense operations

⁠Components:

• App Image: alokdekadev/tokensage-app:latest (659MB)
• LLM Service: ollama/ollama:latest
• Local Model: llama3.2:1b (~1.3GB)
• Cloud Service: OpenAI API (configurable models)

⁠Roadmap:

• Multi-model support (Mistral, Gemma, etc.)
• Advanced token optimization strategies
• Full source code release on GitHub
• Enhanced UI and configuration options

⁠Built With:

Python • Streamlit • DSPy • Ollama • OpenAI API • Docker