dimitri/pgloader repository overview

⁠PGLoader

pgloader is a data loading tool for PostgreSQL, using the COPY command.

Its main advantage over just using COPY or \copy, and over using a Foreign Data Wrapper, is its transaction behaviour, where pgloader will keep a separate file of rejected data, but continue trying to copy good data in your database.

The default PostgreSQL behaviour is transactional, which means that any erroneous line in the input data (file or remote database) will stop the entire bulk load for the table.

pgloader also implements data reformatting, a typical example of that being the transformation of MySQL datestamps 0000-00-00 and 0000-00-00 00:00:00 to PostgreSQL NULL value (because our calendar never had a year zero).

⁠LICENCE

pgloader is available under The PostgreSQL Licence⁠.

⁠Usage

You can either give a command file to pgloader or run it all from the command line, see the pgloader quick start⁠ on https://pgloader.readthedocs.io⁠ for more details.

$ docker run --rm -it dimitri/pgloader:latest pgloader --help
$ docker run --rm -it dimitri/pgloader:latest pgloader --version

For example for a full migration from MySQL, including schema definition (tables, indexes, foreign keys, comments) and parallel loading of the corrected data:

$ docker run --rm -it dimitri/pgloader:latest \
     pgloader \
       mysql://root:[email protected]/dbname \
       pgsql://postgres:[email protected]/dbname

In some cases using a version of pgloader built with Clozure-CL instead of SBCL can be worthwile, so it's available too:

$ docker run --rm -it dimitri/pgloader:ccl.latest pgloader --version