pleosoft/summarize-transformer repository overview

⁠summarize transformation engine

The Summarize engine is a transformer-based service designed to serve as a bridge between your application and the summarization API. It acts as a proxy, forwarding requests to the /summarize/ API, enabling you to easily integrate advanced text summarization into your workflow.

With the flexibility to integrate with any AI service that exposes a similar /summarize/ API, this t-engine can serve as a central point for summarization tasks in your enterprise applications, leveraging transformer models like T5, BART, and potentially any future models with compatible APIs.

⁠example summarize api

on github⁠

make sure to run it along side the summarize transformation engine

or add this to your alfresco docker compose file

  summarize-transformer:
    image: pleosoft/summarize-transformer:1.0.0
    environment:      
      SUMMARIZER_CONFIG_BASE_URL: http://summarize-api:8000

  summarize-api:
    image: dgradecak/summarizer-api:latest

• the summarizer-api is just an example. This is not a perfect AI model but is serving as a base idea for your creativity. Check on the github repo above, clone it and adapt to your needs.
• DO NOT use it in production but make your own.

and update alfresco repository environment property

-DlocalTransform.summarization.url=http://summarize-transformer:8092

• do not forget to create a rule for Alfresco runtime
• once all is setup and working, by default the summary will be in cm:description

⁠config options

defaults:

summarizer.config:
  baseUrl: http://localhost:8000
  maxLength: 100
  minLength: 10
  doSample: false
  textStart: 0
  textEnd: 1000
  property: "{http://www.alfresco.org/model/content/1.0}description"

baseUrl: the base url of the summarize api

⁠max_length (Maximum Length)

• Description: This parameter controls the maximum length of the generated summary.
• Purpose: It ensures that the summary will not exceed the specified number of tokens or characters.
• Use Case: If you want to make sure the summary is concise and doesn't exceed a certain length, you would set this parameter to an appropriate value. For example, setting max_length = 100 ensures that the summary will be at most 100 tokens long.

⁠min_length (Minimum Length)

• Description: This parameter controls the minimum length of the generated summary.
• Purpose: It ensures that the summary will not be shorter than the specified number of tokens or characters.
• Use Case: This can be useful to prevent the model from generating a summary that is too short or lacks important information. For example, setting min_length = 50 ensures the summary will be at least 50 tokens long, even if the original text is very short.

maxLength = 100: The summary will not be longer than 100 tokens. minLength = 50: The summary will be at least 50 tokens long. doSample = false: The summary will be deterministic and the same every time for the same input.

⁠do_sample (Sampling)

• Description: This boolean parameter determines whether to sample from the model's distribution of possible summaries or to generate a deterministic output.
• Purpose: When do_sample is set to True, the model introduces randomness into the summary generation process. This means that every time you run the summarization, you might get a slightly different result, even for the same input text. True: The model will generate diverse summaries by sampling from the distribution of possible outputs. False: The model will generate a deterministic summary, which will be the same every time you run it for the same input text.

⁠text end/start

• Description: this is substracting the text on the summarize engine side before sending to the summarize api textStart: 0 textEnd: 1000

⁠api

This transform engine handles target mimetypes alfresco-metadata-extract and text/plain.

Request Body:

{
  "text": "Your long text here...",
  "max_length": 100,    // Maximum length of the summary (optional)
  "min_length": 10,     // Minimum length of the summary (optional)
  "do_sample": false    // Whether to sample or generate deterministic summaries (optional)
}

Response for text/plain target mimetype:

{
  "summary": "This is the summarized text..."
}

Response for alfresco-metadata-extract target mimetype:

{
  "{http://www.alfresco.org/model/content/1.0}description": "This is the summarized text..."
}

⁠deployment with Alfresco

To deploy this Summarize engine within your Alfresco-based system, follow the standard Alfresco t-engine integration steps. The t-engine will act as a proxy, forwarding text summarization requests to the backend API, which can be configured based on your deployment preferences (e.g., a self-hosted FastAPI service or a third-party API).

⁠add the transformer to alfresco-global.properties

example for a docker installation

localTransform.summarization.url=http://summarization:8092

⁠Alfresco repository pipelines

follow Alfresco t-engines configuration to configure renditions or different pipelines to transform to text and as the last step use summarization

⁠test with Alfresco

if you do not have pipelines or renditions you can only test text/plain source mimetype. And to be able to extract metadata make sure to run the extract metadata action. You could also that by configure rules with the action "Extract common metadata fields".

⁠extending the summarize engine

Since the Summarize engine is built to be flexible, you can extend it to support any other AI service that exposes a /summarize/ API. Simply modify the configuration to point to the new service, and the t-engine will handle the integration, forwarding text summarization requests and returning results seamlessly.

This approach ensures that your summarization needs are met, regardless of the underlying AI model, while maintaining flexibility for future changes.

⁠License

This software is provided free of charge for use, but without any warranty. It is built as a native image on top of Spring Framework libraries and Spring Boot. The software includes no third-party software except for Spring Framework libraries, OpenTelemetry/Micrometer, and proprietary code developed by Pleo Soft d.o.o.

The source code for the Docker image (non-native images) and Windows executables is available to customers upon request. The binaries are distributed under an unlockable source code license, allowing free usage with limitations on access to the source code.