misolims/miso-lims-migration repository overview

⁠Available Tags

• latest Dockerfile⁠

Check the Tags⁠ page for the latest versions.

⁠What is MISO?

MISO (Managing Information for Sequencing Operations) is an open-source Laboratory Information Management System (LIMS) specifically designed for tracking next-generation sequencing experiments.

The miso-lims-migration container is a standalone container that can initialize a new MISO database or migrate your existing database to match a new MISO version. For the full webapp, please see miso-lims-webapp⁠.

⁠How to use this image

In most cases, you shouldn't need to use this image directly. It will be deployed as part of the miso-lims-webapp⁠ docker-compose file. However, if you are testing or need to migrate a local database, full instructions are below.

⁠Environment variables

This connection is configured by passing the following environment variable to the container:

• MISO_DB_HOST_PORT : the hostname and the port to connect to. The hostname can either be a full DNS name, without the protocol e.g. myserver.mycompany.com, or refer to a link to another container. (default: 'db:3306')
• MISO_DB_PASS_FILE : a file with a single line containing the password for the MYSQL user used to query and update the DB. This can be the path to a docker secret (default: /run/secrets/lims_password)
• MISO_DB : the name of the MISO db on the indicated host (default: 'lims')
• MISO_DB_USER : the MySQL user on the host DB (default: tgaclims)

⁠Starting a MISO-compatible MySQL container

We use the MySQL 5.7.25⁠ Docker container in MISO. Other versions are not supported.

For long-term use, we highly recommend setting your database up with a Docker volume⁠ or specifying a local storage location by mounting to /var/lib/mysql.

In this instance, we start a MySQL container named db without any persistent storage. We also mount a local password file .miso_db_password instead of a docker secret.

docker run --rm -it --name db \
             -e MYSQL_RANDOM_ROOT_PASSWORD='yes' \
             -e MYSQL_DATABASE='lims' \
             -e MYSQL_USER='tgaclims' \
             -e MYSQL_PASSWORD_FILE=/.miso_db_password \
             --mount type=bind,source=${PWD}/.miso_db_password,target=/.miso_db_password  \
             -d \
             mysql:5.7.25

⁠Migrating a database

If you already have a MySQL MISO database running, launch the container to perform migration to the latest version of MISO. This relies on there being a text file locally at ./.miso_db_password that contains the password for the MySQL MISO_DB_USER.

⁠Database Backup

Since Flyway does not have a rollback mechanism that leaves the database in a good state, we recommend backing up your database before performing any migration. Use mysqldump⁠ to make a backup copy.

⁠Migrating a database by IP

In this version, we migrate a database lims located at IP and port 172.19.0.1:3306 using the database user tgaclims and the password in the local file ./.miso_db_password to the latest version of the MISO database. There are a lot of WARNINGS. These can be ignored.

docker run --rm -it --name miso-lims-migration \
             -e MISO_DB_USER='tgaclims' \
             -e MISO_DB='lims' \
             -e MISO_DB_HOST_PORT='172.19.0.1:3306' \
             -e MISO_DB_PASS_FILE=/.miso_db_password \
             --mount type=bind,source=${PWD}/.miso_db_password,target=/.miso_db_password  \
             misolims/miso-lims-migration:latest

Successful migrations can be performed repeatedly with no consequences, and each migration version will apply all of the changes between the current DB version and latest.

⁠Migrating a MySQL docker container

Create a bridge network to make sure the data container and the migration container can talk to each other, and add the running MySQL container to it. In this case the container's name is db

docker network create miso-net
docker network connect miso-net db

Then perform the migration, adding --network miso-netto the migration command.

docker run --rm -it --name miso-lims-migration \
             -e MISO_DB_USER='tgaclims' \
             -e MISO_DB='lims' \
             -e MISO_DB_HOST_PORT='db:3306' \
             -e MISO_DB_PASS_FILE=/.miso_db_password \
             --mount type=bind,source=${PWD}/.miso_db_password,target=/.miso_db_password  \
             --network miso-net \
             misolims/miso-lims-migration:latest

⁠Running other flyway commands

The bash script inside the container at /run-flyway (source⁠) can be launched with any available flyway command and will pass the correct URL and credentials to flyway. To see a list of available commands, run the following:

docker run --rm -it \
             -e MISO_DB_USER='tgaclims' \
             -e MISO_DB='lims' \
             -e MISO_DB_HOST_PORT='db:3306' \
             -e MISO_DB_PASS_FILE=/.miso_db_password \
             --mount type=bind,source=${PWD}/.miso_db_password,target=/.miso_db_password  \
             misolims/miso-lims-migration:latest \
             bash /run-flyway --help

⁠Adding custom migrations

Mount a directory to /flyway/migrations/ inside the miso-lims-migration container with your own migrations, named with the usual V000__migration_name.sql Flyway versioned migration syntax⁠.

The migrations in this directory will be applied in numerical order, merged with other MISO migrations in miso-lims/sqlstore/src/main/resources/db/migration⁠. To ensure your migration is run at the end, choose a sufficiently high number, e.g.V9900__mymigration.sql.

In this case, we're launching the docker container and mounting a custom migration in V9900__mymigration.sql. Make sure that you backup your database before doing this, since if the migration fails the database will be in an inconsistent state.

docker run --rm -it \
             -e MISO_DB_USER='tgaclims' \
             -e MISO_DB='lims' \
             -e MISO_DB_HOST_PORT='db:3306' \
             -e MISO_DB_PASS_FILE=/.miso_db_password \
             --mount type=bind,source=${PWD}/.miso_db_password,target=/.miso_db_password  \
             --mount type=bind,source=${PWD}/V9900__mymigration.sql,target=/flyway/migrations/V9900__mymigration.sql \
             misolims/miso-lims-migration:latest

⁠Help

For help using this container, file an issue on the miso-lims issue tracker⁠ or talk to us on our miso-lims Gitter channel⁠.