intel/enterprise-rag-text-compression

Verified Publisher

By Intel Corporation

•Updated 24 days ago

Intel® AI for Enterprise RAG Text Compression Microservice for compressing text from documents

Image

793

Overview Tags

intel/enterprise-rag-text-compression repository overview

⁠Intel® AI for Enterprise RAG Text Compression Microservice

Part of the Intel® AI for Enterprise RAG ecosystem.

⁠🔍 Overview

This microservice is designed to compress text from loaded documents. Result of this microservice can then be passed to Data Splitter microservice and, later, to Embedding microservice and ultimately persisted in the system.

⁠Support Matrix

The microservice supports following compression techniques:

Name	Short description
header_footer_stripper	Removes common boilerplate text like footers, confidentiality notices, and signatures to focus on the meaningful content.
ranking_aware_deduplication	Removes duplicate or near-duplicate content while intelligently preserving the highest-quality version of each segment.

This service integrates with other Intel® AI for Enterprise RAG components:

Text Extractor supplies loaded input data
Text Splitter service for splitting the loaded data
Ingestion service for vector database integration
Embedding service for generating vector representations of documents for semantic search
EDP Enhanced Dataprep
Retriever service for accessing processed documents

⁠License

Intel® AI for Enterprise RAG is licensed under the Apache License, Version 2.0.

Tag summary

Recent tags

Content type

Image

Digest

sha256:159f35383…

Size

117.7 MB

Last updated

24 days ago

docker pull intel/enterprise-rag-text-compression

This week's pulls

Pulls:

Last week

Learn more⁠