navigatorsguild/osm-admin repository overview

⁠osm-admin

⁠Overview

osm-admin is an Open Street Map (OSM)⁠ data administration tool. It provides efficient import and export of *.osm.pbf files into and from an apidb schema on Postgresql database for use by the openstreetmap-website⁠.

⁠Status

The project is in the beta stage of its first version 0.1.0

⁠Issues

Issues are welcome and appreciated. Please submit to https://github.com/navigatorsguild/osm-admin/issues⁠

⁠Roadmap

⁠v0.2.0

• Implement synch of OSM changes from https://planet.openstreetmap.org/⁠ - #1⁠
• merge OSM changes into apidb schema - #2⁠

⁠v0.3.0

• implement option to drop history from export - #3⁠
• index the contents of the DB by S2 Cells - #4⁠
• index the contents by geographical context - #5⁠
• implement regional extracts from OSM database - #6⁠

⁠Usage

This software is distributed as a docker container at https://hub.docker.com/r/navigatorsguild/osm-admin⁠ All the software that powers the container is located in this GitHub repository⁠ and is available under MIT or Apache-2.0 licences

⁠Pull

$ docker pull navigatorsguild/osm-admin

⁠Prepare the data

The data can be downloaded from https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf⁠ by HTTPS or by torrent. To get started we recommend using geographical extracts prepared with osmium or downloaded from http://download.geofabrik.de/⁠

$ osmium getid -r -t planet-latest.osm.pbf r365307 -o malta-boundary-latest.osm
$ osmium extract -p malta-boundary-latest.osm -o malta-latest.osm.pbf planet-latest.osm.pbf

or downloaded from http://download.geofabrik.de/⁠

curl -v -o malta-latest.osm.pbf http://download.geofabrik.de/europe/malta-latest.osm.pbf

Validate the file with osmium

osmium fileinfo -e malta-latest.osm.pbf

Assuming there is a PostgreSQL database named OSM_DATABASE running on OSM_HOST:OSM_PORT with a user OSM_USER the following command will populate the database with the data from freshly downloaded malta-latest.osm.pbf. If there is no readily available database for experimentation, please see the Experiment⁠ page for instructions on how to set up one.

⁠Import

$ touch touch pg_restore.log
$ touch touch pg_restore.error.log
$ docker volume create osm-admin-vol
$ docker run --rm --name osm-admin -it \
  -v ${PWD}/<PGPASSFILE>:/root/.pgpass \
  -v osm-admin-vol:/var/lib/osm/ \
  -v ${PWD}/pg_restore.log:/var/log/osm/pg_restore.log \
  -v ${PWD}/pg_restore.error.log:/var/log/osm/pg_restore.error.log \
  -v ${PWD}malta-latest.osm.pbf:/var/lib/osm/input/malta-latest.osm.pbf \
  navigatorsguild/osm-admin:latest \
  import \
  --verbose \
  --input /var/lib/osm/input/malta-latest.osm.pbf \
  --input-format pbf \
  --output /var/lib/osm/output/malta-latest \
  --host <OSM_HOST> \
  --port <OSM_PORT> \
  --user <OSM_USER> \
  --database <OSM_DATABASE> \
  --no-password

Specifying --pasword will prompt for password. There is an option to use --no-password for trust connections and with pgpass file. Please see an example of PGPASSFILE in ./db/pgpass and the documentation at https://www.postgresql.org/docs/current/libpq-pgpass.html⁠ Please note that the permissions for PGPASSFILE must be 0o600.

⁠Export

$ touch touch pg_dump.log
$ touch touch pg_dump.error.log
$ docker volume create osm-admin-vol
$ docker run --rm --name osm-admin -it \
  -v ${PWD}/<PGPASSFILE>:/root/.pgpass \
  -v osm-admin-vol:/var/lib/osm/ \
  -v ${PWD}/pg_dump.log:/var/log/osm/pg_dump.log \
  -v ${PWD}/pg_dump.error.log:/var/log/osm/pg_dump.error.log \
  -v ${PWD}/output/:/var/lib/osm/output \
  navigatorsguild/osm-admin:latest \
  export \
  --verbose \
  --dump /var/lib/osm/dump \
  --output /var/lib/osm/output/result.osm.pbf \
  --output-format pbf \
  --compression-level 0 \
  --host <OSM_HOST> \
  --port <OSM_PORT> \
  --user <OSM_USER> \
  --password"

⁠Develop

See instructions for setting up the development⁠ environment.

⁠Experiment

See instructions for setting up the experimentation⁠ environment.

⁠Credits

• osmosis⁠ - a powerful OSM data processor.
  • used to bootstrap the development environment
• osmium⁠ - an very fast OSM data processor.
  • used to bootstrap the development environment
• Open Street Map (OSM)⁠
  • apidb⁠ schema definition

⁠Similar projects

• osmosis⁠ - a powerful OSM data processor.
• osmium⁠ - an very fast OSM data processor.

⁠License

MIT OR Apache-2.0