hiddewie/map-it-import

By hiddewie

•Updated about 1 month ago

Scripts for generating custom cycling maps using Mapnik and open data

Image

2.3K

Overview Tags

hiddewie/map-it-import repository overview

⁠Generating a custom cycling map with Mapnik and Carto

View the blogpost describing this project at https://dev.to/hiddewie/creating-a-custom-cycling-map-3g2a⁠.

⁠Cartography and features

Cartography⁠

Features⁠

⁠Getting started

There are three scripts in this repository:

scripts/download.sh⁠

Downloads the required data into a Postgres database for the map.
scripts/generate.py⁠

A Python script which will generate the map for a Mapnik configuration. The Mapnik configuration can be generated by running Carto against carto/project.mml.
scripts/bounds.py⁠

A Python script which will output a list of bounding boxes that will fit the configured page size and bounding box perfectly.

See the environment variables which can be configured for the scripts below.

Copy the distributed environment file .env.dist⁠ to .env. You can modify the .env file to suit your needs.

The scripts are packaged as Docker images, and configured in the file docker-compose.yml⁠.

⁠Database

Start a database with GIS extensions enabled using the image postgis/postgis⁠:

docker-compose -d up postgres-osm

Used technology:

⁠Map bounds

Determine the bounding box of the region you want to print. The bounding box will be used to determine the number of pages to print. If everything fits on one page (of the configured paper size) then padding is added until the page is filled exactly. If the content needs more than one page, then multiple tiled pages are generated to cover the bounding box area, with the configured page overlap (5% by default).

The Map bounds tool⁠ can be used to choose coordinates on a map in a visual manner.

In addition to the visual tool, the script container hiddewie/map-it-bounds⁠ can be used. This commandline tool contains the same logic in the visual tool and as in the map generation script. The output will contain the bounding boxes for each page that will be generated. These values can be used for other commands.

docker-compose run map-it-bounds

(You can also build it yourself using docker-compose build map-it-bounds)

⁠Data download & import

Make sure you have created an account U.S. Geological Survey⁠. The USGS service is used to download terrain height information in high resolution.

Then, download and import the data of the map using the docker image hiddewie/map-it-import⁠. Map the data directory of this project to the container. Some files are downloaded there that are used for shading the map. Run it using

docker-compose run map-it-import

where $USGS_USERNAME and $USGS_PASSWORD are credentials for the U.S. Geological Survey⁠.

(You can also build it yourself using docker-compose build map-it-import)

Used technology:

Phyghtmap⁠
Osmium(-tool)⁠
osm2psql⁠
shp2pgsql, part of PostGIS⁠
GDAL⁠
GeoFabrik⁠ as a data source for OSM data
U.S. Geological Survey⁠ as a data source for elevation data

⁠Map

Let's generate a map. Use the image hiddewie/map-it⁠ and run it using

docker-compose run map-it

The map will be written to the mapped volume in the /output directory. The mapnik XML config will also be written there.

The bounding box does not need to fit perfectly on one page. If it does not, padding will be added or multiple pages will be generated.

(You can also build it yourself using docker-compose build map-it)

Used technology:

⁠Tiles

The same Docker image can also generate tiles in the output folder. Instead of generating a PDF map, these tiles can be used for a sliding online map.

docker-compose run map-it /usr/bin/python3 tiles.py

The tiles will be generated in the output/tiles directory.

Used technology:

CartoCSS⁠
Mapnik⁠
Icons from OpenStreetMap Carto style⁠
Tile rendering script based on generate_tiles.py⁠

⁠Script parameters

The lists below describe the parameters used for the scripts, including defaults.

⁠Import script

PG_HOST (default localhost): The Postgres database host
PG_PORT (default 5432): The Postgres database port
PG_USER (default osm): The Postgres database user
PG_PASSWORD (default empty): The Postgres database password
PG_DATABASE (default gis): The Postgres database
FEATURE_COUNTRIES (required, default empty): Countries that will be downloaded from GeoFabrik⁠. Separated by whitespace. For example europe/netherlands/overijssel europe/slovakia europe/poland/slaskie europe/poland/malopolskie.
USGS_USERNAME and USGS_PASSWORD (required, default empty): Create an account for accessing U.S. Geological Survey⁠ for terrain information. This can be done for free . Set the credentials in these environment variables.
The ⁠phyghtmap⁠ tool is used for downloading terrain information.
BBOX (required, default empty): Of the form A:B:C:D, for example 5.3:51.1:6.8:53.0056 where (A, B) is the lower left corner of the bounding box and (C, D) is the top right corner. Specify in longitude - latitude order in the EPSG:4326⁠ coordinate system.

Optional extra parameters for tweaking the import of downloaded OpenStreetMap data into the database:

OSM2PGSQL_CACHE (default 1024): The cache size in mega bytes that the import script may use.
OSM2PGSQL_NUMPROC (default 4): The number of processes that import script may use.

⁠Map generation & bounding box script

PG_HOST (default localhost): The Postgres database host
PG_PORT (default 5432): The Postgres database port
PG_USER (default osm): The Postgres database user
PG_PASSWORD (default empty): The Postgres database password
PG_DATABASE (default gis): The Postgres database
MAP_NAME (default map): The name of the map. Used for generating filenames. Existing files will be overwritten. The filename will be suffixed with the index of the generated page if more than one page is generated because the bounding box area does not fit on one page.
BBOX (required, default empty): Of the form A:B:C:D, for example 5.3:51.1:6.8:53.0056 where (A, B) is the lower left corner of the bounding box and (C, D) is the top right corner. Specify in longitude - latitude order in the EPSG:4326⁠ coordinate system.
SCALE (default 1:150000): The scale of the map, when printed on the indicated paper size. The value is of the form 1:N with N a number.
PAPER_ORIENTATION (default portrait): The orientation of the generated page. Valid values: portrait and landscape.
PAPER_SIZE (default A4): The size of the generated page. Valid values: A0, A1, A2, A3, A4 and A5, or any value of the form A mm x B mm (millimeters), A in x B in (inches) or A m x B m (meters) with A and B numeric values. For example A1, 10 mm x 100 mm or 20 in x 5 in.
PAGE_OVERLAP (default 5%): A percentage of the form '5%' or '10.1%'. The percentage of each page is taken on all four sides of the paper as padding. When multiple pages are generated the padding will cause overlap between the pages.

⁠Tile generation script

PG_HOST (default localhost): The Postgres database host
PG_PORT (default 5432): The Postgres database port
PG_USER (default osm): The Postgres database user
PG_PASSWORD (default empty): The Postgres database password
PG_DATABASE (default gis): The Postgres database
BBOX (required, default empty): Of the form A:B:C:D, for example 5.3:51.1:6.8:53.0056 where (A, B) is the lower left corner of the bounding box and (C, D) is the top right corner. Specify in longitude - latitude order in the EPSG:4326⁠ coordinate system.
NUM_THREADS (required, default 6): The number of threads that will be used to concurrently render tiles.
MIN_ZOOM and MAX_ZOOM (required, both default 12): The minimum and maximum zoom level to generate.
TMS_SCHEME (true or false, default false): Generate tiles for schemes with the TMS format of the y coordinate.
SKIP_IF_EXISTS (true or false, default true): Whether tile generation should be skipped if the tile file already exists.