dimitri/pgloader
Automatic build of current pgloader master's tree available as a docker image.
100K+
pgloader is a data loading tool for PostgreSQL, using the COPY
command.
Its main advantage over just using COPY
or \copy
, and over using a
Foreign Data Wrapper, is its transaction behaviour, where pgloader
will keep a separate file of rejected data, but continue trying to
copy
good data in your database.
The default PostgreSQL behaviour is transactional, which means that any erroneous line in the input data (file or remote database) will stop the entire bulk load for the table.
pgloader also implements data reformatting, a typical example of that
being the transformation of MySQL datestamps 0000-00-00
and
0000-00-00 00:00:00
to PostgreSQL NULL
value (because our calendar
never had a year zero).
pgloader is available under The PostgreSQL Licence.
You can either give a command file to pgloader or run it all from the command line, see the pgloader quick start on https://pgloader.readthedocs.io for more details.
$ docker run --rm -it dimitri/pgloader:latest pgloader --help
$ docker run --rm -it dimitri/pgloader:latest pgloader --version
For example for a full migration from MySQL, including schema definition (tables, indexes, foreign keys, comments) and parallel loading of the corrected data:
$ docker run --rm -it dimitri/pgloader:latest \
pgloader \
mysql://root:$MYSQL_ROOT_PASSWORD@172.17.0.2/dbname \
pgsql://postgres:$POSTGRES_PASSWORD@172.17.0.3/dbname
In some cases using a version of pgloader built with Clozure-CL instead of SBCL can be worthwile, so it's available too:
$ docker run --rm -it dimitri/pgloader:ccl.latest pgloader --version
docker pull dimitri/pgloader