tolerantsoftware/match

By tolerantsoftware

•Updated 15 days ago

Official Images for TOLERANT Match

Image

Machine learning & AI

Data science

Content management system

2.5K

Overview Tags

tolerantsoftware/match repository overview

⁠Quick reference

Maintained by: https://www.tolerant-software.de/⁠
Where to get help: [email protected]⁠
Release Notes: https://www.tolerant-software.de/en/products/tolerant-match/product-support/⁠
All Configurations and further documentation to use this image orchestrated with other images for the complete Match Experience can be found in our GitHub: https://github.com/tolerantsoftware/match⁠

⁠Supported tags and respective Documentation links

⁠Product Overview

⁠Duplicate matching, data cleansing and address cleansing for address lists and customer data

Well-maintained address lists and customer master data are the basis for successful marketing and sales campaigns. In addition, an error-free address database is crucial for meaningful analyses and statistics. This is the only way to make operational and analytical decisions that drive your business forward. Therefore, it is important to perform a duplicate check on a regular basis to find duplicates and clean up addresses. With TL Match you can find these duplicates in an automated, accurate and error-tolerant way.

⁠Fast and error-tolerant search

TL Match finds address data, products and customers quickly and accurately even in large data sets and even if only fragmentary information is available. The special technology in TL Match also allows you to combine different search options.

⁠Merge customer data from different sources in a clean and error-tolerant way

Legal requirements for data protection make it increasingly difficult to use external data sources for customer data. This makes it all the more important to maintain the high data quality of your own address databases. Addresses must be correct: Only if your customer data is clean can you utilise and deploy it effectively. But the volume of data is inevitably growing. And often customer data has to be maintained and matched in different systems or databases in parallel. With TL Match you can easily merge data from different sources without duplicates.

⁠Accurately find duplicates with TL Match

TL Match finds what belongs together. The easy-to-use software works accurately and error-tolerantly. Powerful search algorithms find data records that match already during data entry, but also during subsequent address cleansing or matching from multiple sources. TL Match’s error-tolerant search takes into account spelling or typing errors in names and addresses as well as different spellings. In addition, you can adapt TL Match to your needs and requirements at any time. For best results, clean data and duplicate-free address databases.

Some examples

RECORD	NAME	STREET	POSTAL CODE	TOWN/CITY
query	Alex Schepanski	Margretenstr. 9		Muenchen
result	Alexander Sczepansky	Margretenstr. 7	80331	München
query	Christina Maier	Bachstr. 1		Weissach
result	Kristina Meyer	Bachstr. 1	71287	Weissach

⁠TOLERANT Match image

⁠Usage

Notes

The container needs licenses to run properly. For proper test licences please contact [email protected]⁠.

The required memory depends on the amount of data used.

The required configuration files can be found on the official Match GitHub⁠

Starting and stopping can be managed using a simple docker compose file. Here is an example without security:

services:

  backend:
    image: ${TMATCH_DOCKER_REGISTRY}/match:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 6g
    mem_reservation: 2g
    cpus: 4
    # Uncomment the below line and set user and group id (from host system) to use it in container
    # user: "$UID:$GID"
    volumes:
      # persist match logs
      - match-logs:/opt/tolerant/logs
      # Example for custom configuration via volume:
      - match-config:/opt/tolerant/config
      # Example for custom data via volume
      - match-data:/opt/tolerant/data
      # Example for custom protocols via volume
      - match-protocols:/opt/tolerant/protocols
    environment:
      # service id
      - TLMATCH_SERVICE_ID=${TLMATCH_SERVICE_ID}
      # number of processors to be used by the JVM
      - JVM_OPT_ACTIVE_PROCESSOR_COUNT=4
      - TZ=${TZ}
      # If no prometheus export is wanted, set to false
      - TL_PROMETHEUS_METRICS_ENABLED=true
      # comment this line in to change the logging profile. possible values: anonymized, console, request, jdbc or multi-project
      # - LOGGING_PROFILE=console
      # if no gui is wanted, set to false
      - GUI_ENABLED=true

  gui:
    image: ${TMATCH_DOCKER_REGISTRY}/match-gui:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 2g
    mem_reservation: 512M
    environment:
      - GUI_BASE_PATH=/matchService
      - API_BASE_PATH=/matchService
    depends_on:
      backend:
        condition: service_healthy

  prometheus:
    image: prom/prometheus:${PROMETHEUS_VERSION}
    volumes:
      - type: bind
        source: ./config/prometheus/prometheus.yml
        target: /etc/prometheus/prometheus.yml
        read_only: true
      - prometheus-data:/prometheus
    expose:
      - 9090
    depends_on:
      - backend

  grafana:
    image: grafana/grafana:${GRAFANA_VERSION}
    volumes:
      - ./config/grafana/datasources:/etc/grafana/provisioning/datasources
      - ./config/grafana/dashboards/service:/etc/grafana/provisioning/dashboards
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_USER=tolerant
      - GF_SECURITY_ADMIN_PASSWORD=tolerant
    ports:
      - ${GRAFANA_PORT}:3000
    depends_on:
      - prometheus

  proxy:
    image: nginx:${NGINX_VERSION}-alpine
    restart: unless-stopped
    volumes:
      - type: bind
        source: ./config/nginx/common.loc.template
        target: /etc/nginx/templates/common.loc.template
        read_only: true
      - type: bind
        source: ./config/nginx/default.conf.template
        target: /etc/nginx/templates/default.conf.template
        read_only: true

      # persist nginx logs
      - nginx-logs:/etc/nginx/logs
    environment:
      - PROXY_HOST_PORT=${NGINX_PORT}
      - GUI_BASE_PATH=/matchService
      - SERVER_CONTEXT_PATH=/matchService

    ports:
      - ${NGINX_PORT}:8080
    depends_on:
      backend:
        condition: service_started
      gui:
        condition: service_healthy

volumes:
  # Volume for persistent data of keycloak
  nginx-logs:
  match-logs:
  grafana-data:
  prometheus-data:
  match-protocols:
  # Examples for external created volumes with custom configuration and data
  match-config:
  #   external: true
  match-data:
  #   external: true

Here is an example with enabled security:

services:

  backend:
    image: ${TMATCH_DOCKER_REGISTRY}/match:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 6g
    mem_reservation: 2g
    cpus: 4
    # Uncomment the below line and set user and group id (from host system) to use it in container
    # user: "$UID:$GID"
    volumes:
      # persist post logs
      - match-logs:/opt/tolerant/logs
      # Example for custom configuration via volume:
      - match-config:/opt/tolerant/config
      # Example for custom data via volume
      - match-data:/opt/tolerant/data
      # Example for custom protocols via volume
      - match-protocols:/opt/tolerant/protocols
    environment:
      - TLMATCH_SERVICE_ID=${TLMATCH_SERVICE_ID}
      - INTERNAL_IDENTITY_PROVIDER_URL=${INTERNAL_IDENTITY_PROVIDER_URL}
      - IDENTITY_PROVIDER_URL=${IDENTITY_PROVIDER_URL}
      - TOLERANT_CLIENT_ID=${TOLERANT_CLIENT_ID}
      - SECURITY_ENABLED=true
      - TOLERANT_REALM=${TOLERANT_REALM}
      # number of processors to be used by the JVM
      - JVM_OPT_ACTIVE_PROCESSOR_COUNT=4
      - TZ=${TZ}
      # If no prometheus export is wanted, set to false
      - TL_PROMETHEUS_METRICS_ENABLED=true
      # comment this line in to change the logging profile. possible values: anonymized, console, request, jdbc or multi-project
      # - LOGGING_PROFILE=console
      # if no gui is wanted, set to false
      - GUI_ENABLED=true
    extra_hosts:
      - host.docker.internal:host-gateway
    depends_on:
      - keycloak

  gui:
    image: ${TMATCH_DOCKER_REGISTRY}/match-gui:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 2g
    mem_reservation: 512M
    environment:
      - GUI_BASE_PATH=/matchService
      - API_BASE_PATH=/matchService
    depends_on:
      backend:
        condition: service_healthy

  prometheus:
    image: prom/prometheus:${PROMETHEUS_VERSION}
    volumes:
      - type: bind
        source: ./config/prometheus/prometheus.yml
        target: /etc/prometheus/prometheus.yml
        read_only: true
      - prometheus-data:/prometheus
    expose:
      - 9090
    depends_on:
      - backend

  grafana:
    image: grafana/grafana:${GRAFANA_VERSION}
    volumes:
      - ./config/grafana/datasources:/etc/grafana/provisioning/datasources
      - ./config/grafana/dashboards/service:/etc/grafana/provisioning/dashboards
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_USER=tolerant
      - GF_SECURITY_ADMIN_PASSWORD=tolerant
    ports:
      - ${GRAFANA_PORT}:3000
    depends_on:
      - prometheus

  openssl:
    image: tolerant-products/openssl
    build: ./config/openssl
    environment:
      - ORGANISATION_UNIT=TOLERANT Match
    volumes:
      - ./config/nginx/ssl/certs:/etc/ssl/certs
      - ./config/nginx/ssl/private:/etc/ssl/private
  proxy:
    image: nginx:${NGINX_VERSION}-alpine
    restart: unless-stopped
    volumes_from:
      - openssl
    volumes:
      - type: bind
        source: ./config/nginx/ssl.conf.template
        target: /etc/nginx/templates/ssl.conf.template
        read_only: true
      - type: bind
        source: ./config/nginx/locations/keycloak.loc.template
        target: /etc/nginx/templates/keycloak.loc.template
        read_only: true
      - type: bind
        source: ./config/nginx/common.loc.template
        target: /etc/nginx/templates/common.loc.template
        read_only: true
      # for case of own ssl certificate
      #- ./config/nginx/ssl/certs:/etc/ssl/certs
      #- ./config/nginx/ssl/private:/etc/ssl/private

    environment:
      - PROXY_HOST_HTTPS_PORT=${NGINX_HTTPS_PORT}
      - CERT_PRIVATE_KEY_FILENAME=${CERT_PRIVATE_KEY_FILENAME}
      - CERT_FILENAME=${CERT_FILENAME}
      # adjust the name of the server to your server name
      - NGINX_SERVER_NAME=${NGINX_SERVER_NAME}
      - TOLERANT_REALM=${TOLERANT_REALM}
      # the following 2 variables are only needed if the included keycloak is to be used
      - INTERNAL_IDENTITY_PROVIDER_URL=${INTERNAL_IDENTITY_PROVIDER_URL}
      - INTERNAL_IDENTITY_PROVIDER_PORT=${KC_PORT}
      - GUI_BASE_PATH=/matchService
      - SERVER_CONTEXT_PATH=/matchService
    ports:
      - ${NGINX_HTTPS_PORT}:8443
    depends_on:
      openssl:
        condition: service_completed_successfully
        restart: true
    extra_hosts:
      - host.docker.internal:host-gateway

  postgres:
    image: postgres:${POSTGRES_VERSION}
    restart: unless-stopped
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: keycloak
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: password

  keycloak:
    image: keycloak/keycloak:${KC_VERSION}
    command: start-dev --import-realm
    restart: unless-stopped
    volumes:
      - type: bind
        source: ./config/keycloak/import
        target: /opt/keycloak/data/import
        read_only: true
    environment:
      - KC_DB=postgres
      - KC_DB_SCHEMA=public
      - KC_DB_USERNAME=keycloak
      - KC_DB_PASSWORD=password
      - KC_DB_URL_HOST=postgres
      - KC_DB_URL_PORT=5432
      - KC_BOOTSTRAP_ADMIN_USERNAME=admin
      - KC_BOOTSTRAP_ADMIN_PASSWORD=admin
      - KC_HOSTNAME_STRICT=false
      - KC_HTTP_ENABLED=true
      - TOLERANT_CLIENT_ID=${TOLERANT_CLIENT_ID}
      - TOLERANT_REALM=${TOLERANT_REALM}
    ports:
      - ${KC_PORT}:8080
    depends_on:
      - postgres

volumes:
  # Volume for persistent data of keycloak
  nginx-logs:
  match-logs:
  grafana-data:
  prometheus-data:
  match-protocols:
  # Examples for external created volumes with custom configuration and data
  match-config:
  #   external: true
  match-data:
  #   external: true
  # Volume for persistent data of keycloak
  postgres-data:

The services can be started using the following commands:

Without security:

docker compose up -d

With enabled security:

docker compose -f compose-secure.yml up -d

The docker compose command should be executed from the directory containing the compose.yaml or compose-secure.yml file.

⁠Stopping

The running services can be stopped using the following commands:

Without security:

docker compose down

With enabled security:

docker compose -f compose-secure.yml down

⁠Starting a batch process

Note The config file and the data must exist inside the container.

A batch process can be stated with the following example

docker compose -f compose-batch.yml up -d

Removing the container after it has exited

docker compose -f compose-batch.yml down

Steps to use your own configuration and data for a batch process

mount your configuration and data to the batch container
adjust the entry point of the compose-batch.yml to match the following pattern:

entrypoint: ["matchBatch.sh", "<configFilename>", "<projectId>"]

⁠Starting an initial load process

Note The Match backend container must be running to execute this command.
The config file and the data must exist inside the container.

An initial load process can be started using the following command:

Without security:

docker compose exec backend service.sh --endpoint operations --function stop.project --parameter projectId=<projectId>
docker compose exec backend matchInitialLoad.sh [-delete-backlog|-keep-backlog] <configFilename> <projectId>
docker compose exec backend service.sh --endpoint operations --function start.project --parameter projectId=<projectId>

With enabled security:

docker compose -f compose-secure.yml exec backend service.sh --endpoint operations --function stop.project --parameter projectId=<projectId>
docker compose -f compose-secure.yml exec backend matchInitialLoad.sh [-delete-backlog|-keep-backlog] <configFilename> <projectId>
docker compose -f compose-secure.yml exec backend service.sh --endpoint operations --function start.project --parameter projectId=<projectId>

The initial load example can be started using the following command:

docker compose exec backend service.sh --endpoint operations --function stop.project --parameter projectId=matchProject-1
docker compose exec backend matchInitialLoad.sh [-delete-backlog|-keep-backlog] config/matchserviceconfig.xml matchProject-1
docker compose exec backend service.sh --endpoint operations --function start.project --parameter projectId=matchProject-1

⁠Match as a cluster

Match can be used in a cluster setup for redundancy purposes. An example compose file for a cluster setup:

services:

  node1:
    image: ${TMATCH_DOCKER_REGISTRY}/match:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 6g
    mem_reservation: 512M
    cpus: 4
    volumes:
      # persist match logs
      - match-logs-node1:/opt/tolerant/logs
      # Example for presentation purposes only:
      - type: bind
        source: ./config/node1/matchserviceconfig.xml
        target: /opt/tolerant/examples/match_service_example/config/matchserviceconfig.xml
      # Example for custom configuration via volume:
      - match-config-node1:/opt/tolerant/config
      # Example for custom data via volume
      - match-data-node1:/opt/tolerant/data
      # Example for custom protocols via volume
      - match-protocols-node1:/opt/tolerant/protocols
    environment:
      # service id
      - TLMATCH_SERVICE_ID=node1
      # number of processors to be used by the JVM
      - JVM_OPT_ACTIVE_PROCESSOR_COUNT=4
      - TZ=${TZ}
      # If no prometheus export is wanted, set to false
      - TL_PROMETHEUS_METRICS_ENABLED=true
      - GUI_ENABLED=true

  node2:
    image: ${TMATCH_DOCKER_REGISTRY}/match:${TMATCH_VERSION}
    restart: unless-stopped
    mem_limit: 6g
    mem_reservation: 512M
    cpus: 4
    volumes:
      # persist match logs
      - match-logs-node2:/opt/tolerant/logs
      # Example for presentation purposes only:
      - type: bind
        source: ./config/node2/matchserviceconfig.xml
        target: /opt/tolerant/examples/match_service_example/config/matchserviceconfig.xml
      # Example for custom configuration via volume:
      - match-config-node2:/opt/tolerant/config
      # Example for custom data via volume
      - match-data-node2:/opt/tolerant/data
      # Example for custom protocols via volume
      - match-protocols-node2:/opt/tolerant/protocols
    environment:
      # service id
      - TLMATCH_SERVICE_ID=node2
      # number of processors to be used by the JVM
      - JVM_OPT_ACTIVE_PROCESSOR_COUNT=4
      - TZ=${TZ}
      # If no prometheus export is wanted, set to false
      - TL_PROMETHEUS_METRICS_ENABLED=true
      - GUI_ENABLED=true

  prometheus:
    image: prom/prometheus:${PROMETHEUS_VERSION}
    volumes:
      - type: bind
        source: ./config/prometheus/prometheus-cluster.yml
        target: /etc/prometheus/prometheus.yml
        read_only: true
      - prometheus-data:/prometheus
    expose:
      - 9090
    depends_on:
      - node1
      - node2

  grafana:
    image: grafana/grafana:${GRAFANA_VERSION}
    volumes:
      - ./config/grafana/datasources:/etc/grafana/provisioning/datasources
      - ./config/grafana/dashboards/cluster:/etc/grafana/provisioning/dashboards
      - grafana-data:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_USER=tolerant
      - GF_SECURITY_ADMIN_PASSWORD=tolerant
    ports:
      - ${GRAFANA_PORT}:3000
    depends_on:
      - prometheus

  gui:
    image: ${TMATCH_DOCKER_REGISTRY}/match-gui:${TMATCH_VERSION}
    restart: unless-stopped
    environment:
      - GUI_BASE_PATH=/matchService
      - API_BASE_PATH=/matchService
    depends_on:
      node1:
        condition: service_healthy
      node2:
        condition: service_healthy

  proxy:
    image: nginx:${NGINX_VERSION}-alpine
    restart: unless-stopped
    volumes:
      - type: bind
        source: ./config/nginx/default_cluster.conf.template
        target: /etc/nginx/templates/default.conf.template
        read_only: true
      - type: bind
        source: ./config/nginx/common.loc.template
        target: /etc/nginx/templates/common.loc.template
        read_only: true
      # persist nginx logs
      - nginx-logs:/etc/nginx/logs
      # Example for ssl data in external volume
      # nginx-ssl:/etc/nginx/ssl
      # In case of basic auth, comment the following in
      # - type: bind
      #   source: ./config/nginx/htpasswd
      #   target: /etc/nginx/htpasswd
      #   read_only: true
    environment:
      - PROXY_HOST_PORT=${NGINX_PORT}
      - SERVER_CONTEXT_PATH=/matchService
      - GUI_BASE_PATH=/matchService
    ports:
      - ${NGINX_PORT}:8080 # If changed, PROXY_HOST_PORT needs to be adjusted to
    depends_on:
      gui:
        condition: service_healthy
      node1:
        condition: service_healthy
      node2:
        condition: service_healthy

volumes:
  nginx-logs:
  match-logs-node1:
  match-logs-node2:
  match-protocols-node1:
  match-protocols-node2:
  grafana-data:
  prometheus-data:
  # Examples for external created volumes with custom configuration and data
  match-config-node1:
  #   external: true
  match-config-node2:
  #   external: true
  match-data-node1:
  #   external: true
  match-data-node2:
  #   external: true

A Match cluster can be started with the following command:

docker compose -f compose-cluster.yml up -d

The docker compose command should be executed from the directory containing the compose-cluster.yml file, otherwise use the -f parameter for specifying the filepath.

A Match cluster can be stopped with the following command:

docker compose -f compose-cluster.yml down

The docker compose command should be executed from the directory containing the compose-cluster.yml file, otherwise use the -f parameter for specifying the filepath.

⁠Match in kubernetes (helm chart)

Match can be used in a kubernetes setup for zero Downtime and Redundancy purposes.

For more information and examples refer to our GitHub repository Match GitHub⁠

⁠License

TOLERANT Match is licensed under a commercial license. This means that the software can only be used for commercial purposes, and that the user must have a valid license to use the software. To get a valid license, please contact [email protected]⁠

Tag summary

Recent tags

Content type

Image

Digest

sha256:a646d2c88…

Size

293.9 MB

Last updated

15 days ago

docker pull tolerantsoftware/match