Sign inSign up

folioci/mod-marc-migrations

By folioci

•Updated 6 days ago

Module designed for high-performance migration of MARC records.

Image
0

10K+

folioci/mod-marc-migrations repository overview

⁠FOLIO - mod-marc-migrations

Module designed for high-performance migration of MARC records.

Code Repository: https://github.com/folio-org/mod-marc-migrations⁠

⁠Metadata

  • Container memory (bytes): 536870912
  • Module ports: 8081/tcp

⁠Default environment variables

  • JAVA_OPTIONS:
    • -XX:MaxRAMPercentage=85.0
  • FOLIO database connection: true
  • FOLIO database common environment:
    • DB_DATABASE, DB_HOST, DB_PORT, DB_USERNAME, DB_PASSWORD, DB_CHARSET
  • FOLIO database other environment:
    • DB_CONNECTION_TIMEOUT
      • Description: This property controls the maximum number of milliseconds that a client will wait for a connection from the pool
      • Default value: 30000
    • DB_IDLE_TIMEOUT
      • Description: This property controls the maximum amount of time that a connection is allowed to sit idle in the pool. This setting only applies when DB_MINIMUM_IDLE is defined to be less than DB_MAXPOOLSIZE
      • Default value: 600000
    • DB_INITIALIZATION_FAIL_TIMEOUT
      • Description: This property controls whether the pool will "fail fast" if the pool cannot be seeded with an initial connection successfully
      • Default value: 30000
    • DB_KEEPALIVE_TIME
      • Description: This property controls how frequently HikariCP will attempt to keep a connection alive, in order to prevent it from being timed out by the database or network infrastructure (0 - disabled)
      • Default value: 0
    • DB_LEAK_DETECTION_THRESHOLD
      • Description: This property controls the amount of time that a connection can be out of the pool before a message is logged indicating a possible connection leak (0 - disabled)
      • Default value: 30000
    • DB_MAXPOOLSIZE
      • Description: This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections
      • Default value: 10
    • DB_MAX_LIFETIME
      • Description: This property controls the maximum lifetime of a connection in the pool
      • Default value: 1800000
    • DB_MINIMUM_IDLE
      • Description: This property controls the minimum number of idle connections that HikariCP tries to maintain in the pool
      • Default value: 10
    • DB_VALIDATION_TIMEOUT
      • Description: This property controls the maximum amount of time that a connection will be tested for aliveness. This value must be less than the DB_CONNECTION_TIMEOUT
      • Default value: 5000
  • CHUNK_FETCH_IDS_COUNT
    • Description: Number of record ids to fetch per query on chunks preparation phase. RECORDS_CHUNK_SIZE should be a divisor for this in order to maintain proper chunk size
    • Default value: 500
  • CHUNK_PERSIST_COUNT
    • Description: Number of chunks to be constructed before persisting to db
    • Default value: 1_000
  • CHUNK_PROCESSING_MAX_PARALLELISM
    • Description: Max thread pool size for chunks processing
    • Default value: 4
  • CHUNK_RETRYING_MAX_IDS_COUNT
    • Description: Maximum number of chunk IDs for retrying
    • Default value: 1000
  • ENV
    • Description: The logical name of the deployment, must be unique across all environments using the same shared Kafka/Elasticsearch clusters, a-z (any case), 0-9, -, _ symbols only allowed
    • Default value: folio
  • OKAPI_URL
    • Description: Okapi URL
    • Default value: okapi
  • RECORDS_CHUNK_SIZE
    • Description: Number of records in one chunk for operation processing
    • Default value: 500
  • S3_ACCESS_KEY_ID
    • Description: S3 compatible service access key
  • S3_BUCKET
    • Description: S3 compatible service bucket
  • S3_IS_AWS
    • Description: Specify if AWS S3 is used as files storage
    • Default value: false
  • S3_LOCAL_SUB_PATH
    • Description: S3 subpath for local storage of Authority and Marc bib files during migration
    • Default value: mod-marc-migrations
  • S3_REGION
    • Description: S3 compatible service region
    • Default value: us-west-2
  • S3_RETRY_COUNT
    • Description: Specify number of retries if S3 client return any kind of error
    • Default value: 3
  • S3_RETRY_DELAY_MS
    • Description: Specify millisecond delay between retries if S3 client return any kind of error
    • Default value: 500
  • S3_SECRET_ACCESS_KEY
    • Description: S3 compatible service secret key
  • S3_SUB_PATH
    • Description: S3 subpath for files storage
    • Default value: mod-marc-migrations
  • S3_URL
    • Description: S3 compatible service url
    • Default value: https://s3.amazonaws.com

Tag summary

Content type

Image

Digest

sha256:2a5a5d3fd…

Size

190.1 MB

Last updated

6 days ago

docker pull folioci/mod-marc-migrations