Sign inSign up

opea/edgecraftrag

By opea

•Updated 6 months ago

Edge Craft RAG (EC-RAG) gateway. Provides a customizable, production-ready retrieval-enhanced genera

Image
0

6.6K

opea/edgecraftrag repository overview

⁠Edge Craft Retrieval-Augmented Generation Application

Edge Craft RAG (EC-RAG) is a customizable, tunable and production-ready Retrieval-Augmented Generation system for edge solutions. It is designed to curate the RAG pipeline to meet hardware requirements at edge with guaranteed quality and performance.

⁠What's New

  1. Support Agent component and enable deep_search agent
  2. Optimize pipeline execution performance with asynchronous api
  3. Support session list display in UI
  4. Support vllm-based embedding service

⁠Table of contents

  1. Architecture⁠
  2. Deployment Options⁠

⁠Architecture

The architecture of the Edge Craft Retrieval-Augmented Generation Application is illustrated below:

flowchart TD
    EC_RAG_UI[EC-RAG UI]
    EC_RAG_Gateway[EC-RAG Gateway]
    Vector_DB[(Vector DB)]
    LLM[[LLM]]

    %% Mega Service 组
    subgraph MegaService["Mega Service"]
        subgraph EC_RAG_Pipeline["EC-RAG Pipeline"]
            %% Indexing 线
            subgraph Indexing["Indexing"]
                Preprocessor[Preprocessor]
                Node_Parser[Node Parser]
                Indexer[Indexer]
            end

            %% Inference 线
            subgraph Inference["Inference"]
                Retriever[Retriever]
                Postprocessor[Postprocessor]
                Generator[Generator]
            end

            Knowledge_Base[(Knowledge Base)]
            Benchmark_Hook[Benchmark Hook]
        end
    end

    %% UI <-> Gateway
    EC_RAG_UI <--> EC_RAG_Gateway

    %% UI -> Pipeline (Configure/Indexing)
    EC_RAG_UI -. Configure .-> EC_RAG_Pipeline
    EC_RAG_UI -->|Indexing| EC_RAG_Pipeline

    %% Gateway -> Pipeline (Inference)
    EC_RAG_Gateway -->|Inference| MegaService

    Preprocessor --> Node_Parser
    Node_Parser --> Indexer
    Indexer --> Knowledge_Base

    Retriever --> Postprocessor
    Postprocessor --> Generator

    Knowledge_Base --> Vector_DB
    Indexer --> Vector_DB

    Generator -->|Inference| LLM
    Retriever -. Configure .-> LLM
    Postprocessor -. Configure .-> LLM

    %% Benchmark Hook
    Benchmark_Hook -.-> Generator

    classDef external fill:#f9f,stroke:#333
    classDef storage fill:#bbf,stroke:#66c
    classDef process fill:#dfd,stroke:#090
    classDef config stroke-dasharray: 5 5

    class EC_RAG_UI,EC_RAG_Gateway,LLM external
    class Vector_DB,Knowledge_Base storage
    class Preprocessor,Node_Parser,Indexer,Retriever,Postprocessor,Generator,Benchmark_Hook process
    class Configure,config config

⁠Deployment Options

The table below lists the available deployment options and their implementation details for different hardware platforms.

PlatformDeployment MethodLink
Intel ArcDocker composeDeployment on Arc⁠

⁠Validated Configurations

Deploy MethodLLM EngineLLM ModelHardware
Docker ComposevLLMQwen3-8BIntel Arc

Tag summary

Content type

Image

Digest

sha256:685db2d79…

Size

159.3 MB

Last updated

6 months ago

docker pull opea/edgecraftrag