b1tw153/overpass-api repository overview

⁠Overpass-API

This project is a fork of drolbr/Overpass-API which is an API to perform queries and analytical processing on OpenStreetMap data.

• This project: b1tw153/Overpass-API⁠
• Upstream: drolbr/Overpass-API⁠
• Overpass API Wiki⁠
• Overpass API Documentation⁠
• Overpass Releases⁠

NOTE: If you're reading this file on Docker Hub, the contents may be truncated. The complete original README.md⁠ file is available in the GitHub repository.

⁠Improvements

• IMPROVED: More resilient replication downloads using fetch_osc.sh or fetch_osc_and_apply.sh
• IMPROVED: Safer recovery from uncontrolled shutdowns in fetch_osc.sh, fetch_osc_and_apply.sh, and apply_osc_to_db.sh
• IMPROVED: Safer controlled shutdown in apply_osc_to_db.sh to reduce the risk of database corruption
• IMPROVED: Faster downloads in fetch_osc.sh, fetch_osc_and_apply.sh, and download_clone.sh
• IMPROVED: Support and optimization for hourly and daily replication sources in fetch_osc.sh, fetch_osc_and_apply.sh, and apply_osc_to_db.sh
• IMPROVED: Adaptive and customizable area creation scheduling in rules_loop.sh
• NEW: Backup script that does not require the server to shut down in backup.sh and an associated restore.sh for recovery
• NEW: Automated cleanup of old diff data in clean_osc.sh
• NEW: Consolidated startup/entrypoint script in run_osm3s.sh
  • Supports minutely/hourly/daily replication
  • Supports attic/meta/basic data sets
  • Supports scheduled area creation
  • Automatically rotates log and output files
  • Automatically remotes old diff data
  • Automatically runs periodic backups
  • Performs periodic health checks on processes
  • Runs on bare metal or as a container entrypoint
• NEW: Automated database initialization from planet and extract files in import_osm_data.sh
  • Supports .osm/.bz2/.pbf data files
  • Supports torrent downloads (when aria2c is available)
  • Automatically sets the starting replicate_id from the replication source
• NEW: Support for container image builds
  • Minimal base image using debian:bookworm-slim
  • Preconfigured with nginx and fcgiwrap
  • External mounts for database, replication, and backup data
  • Control all runtime parameters using environment variables
• NEW: Container status endpoint via container_status.sh that reports the operational status of all Overpass components in plain text or JSON
• NEW: Container image automatically published to Docker Hub as b1tw153/overpass-api⁠
• NEW: Container serves static HTTP content from /static directory
• NEW: Default robots.txt and llms.txt files in /static to guide bot traffic
• NEW: Container serves HTTP 429 and HTTP 504 error responses with clear messaging on backoff and retries
• NEW: Container has a configurable queue to hold requests in nginx when query workers are busy
• NEW: Optional per-client connection and request-rate limiting in nginx (NGINX_CLIENT_CONN_LIMIT, NGINX_CLIENT_REQ_RATE)
• NEW: Bundled Munin node with plugins for nginx, dispatcher, interpreter, and cgroup metrics; opt-in and disabled by default. See src/munin/README.md⁠

⁠Overview

This fork improves the project shell scripts to improve performance, resilience, and flexibility. It also adds support for Docker so that the project can be built as a container image.

There are several useful branches in this repo:

• main: Main branch for the container build and source code for local builds
• master: Tracks the latest release in the upstream drolbr/Overpass-API repo
• release/v0.*: Track the latest revision/hotfix from dev.overpass-api.de⁠ (not available in the upstream repo)

If you plan to run Overpass directly in the OS or build a container image, use the main branch.

If you prefer to build Overpass from the original source, use either the tarballs from the Overpass release web site⁠ or the release/v0.* branches in this fork.

⁠Prerequisites / Requirements

See the basic system requirements⁠ in the Overpass Wiki page.

The container build in this fork includes all of the necessary software. A container management or orchestration system such as Docker Compose or Kubernetes is recommended.

⁠Resource Sizing

⁠Disk

The Overpass database is significantly larger than the source file. For a full planet import, expect roughly 4-5× expansion from the compressed .osm.bz file to the database. Check the current planet file size on planet.openstreetmap.org⁠ before provisioning storage.

Approximate full planet database sizes by metadata mode as of Q2 2026:

Note: the OpenStreetMap planet has grown significantly over time. Figures from older documentation will be substantially lower than current reality.

For extracts, the database will be proportionally smaller, but the expansion ratio from source to database is similar. Check the size of your chosen extract file and apply a 4-5× multiplier as a planning estimate. Note that most extract sources do not include full history, so attic mode is generally not available for extracts; meta mode is typical.

If you enable backups, the backup directory will mirror the database size. Account for this in your storage planning — ideally on a separate storage device.

The diff directory is small and transient. Under normal operation with replication keeping up, it stays well under 1 GiB.

Use fast SSDs. Overpass query performance is heavily dependent on random I/O. Slow disks will significantly degrade query response times under any meaningful load. Ideally, the database files should be on fast NVMe.

⁠CPU

The container runs one fcgiwrap worker per CPU core plus one by default (FCGIWRAP_WORKERS defaults to cpu.max/nproc + 1). Each worker handles one concurrent query. Provision CPU cores based on your expected concurrent query load. A single core is sufficient for light personal use; a public-facing instance benefits from more cores to handle concurrent requests without queuing or rate limiting. An optional request queue can be configured in nginx. See Rate Limiting below.

⁠Memory

Baseline memory usage at idle is negligible — under 50 MiB regardless of database size. Memory pressure comes from query load: each running query allocates memory for result sets and database buffer caches, and complex queries against a full planet database can consume several GiB.

The dispatcher --space limit is mainly a concurrency limit, not a memory constraint. The container automatically sets the dispatcher memory limit to 768GiB to disable this limit and relies on the fcgiwrap worker pool in nginx to constrain concurrency. (You can override this by setting DISPATCHER_BASE_SPACE.) Set your container memory limit based on your expected query workload and monitor actual usage under load to tune it. As a starting point:

• A minimal development instance with a small extract can run with 1-2 GiB
• A lightly loaded personal instance can run with 4-8 GiB
• A moderately loaded instance with complex queries benefits from 16-32 GiB
• A heavily loaded public instance may need 64 GiB or more

Set the container memory limit explicitly — for example, with docker run --memory=16g. Without a memory limit, the container can consume all available host memory under heavy query load.

⁠Installation

⁠Using the Container Image from Docker Hub

The main branch is automatically built and published to Docker Hub as b1tw153/overpass-api:latest⁠. You can pull the latest image or a specific version and use the image directly with the instructions in this README:

docker pull b1tw153/overpass-api:latest

⁠Building Overpass from Source

If you plan to build the Overpass components directly from the source code, there are instructions and guides from several sources:

• Overpass API Wiki⁠
• Overpass API Quick Installation Guide⁠
• Overpass API Complete Installation Guide⁠
• ZeLonewolf/Overpass Installation Guide⁠
• How to Build a Personal Overpass Server on a Tiny Budget⁠
• Setting up an Overpass API server - how hard can it be?⁠

⁠Building the Container Image

Start with the main branch. In the repo directory, run:

docker build -t b1tw153/overpass-api .

The build process will compile the source code. This may take 10-20 minutes.

⁠Initializing the Database

The Overpass API requires a set of database files to run. There are several options to obtain an initial set of database files.

If you're using the container image with bind-mounted host directories, create them and set ownership before running any container commands that write to them. The container runs as uid/gid 10001.

OVERPASS_DB_DIR=      # path to your Overpass database directory on the host
OVERPASS_BACKUP_DIR=  # path to your Overpass backup directory on the host (optional)

sudo groupadd -g 10001 overpass
sudo useradd -u 10001 -s /usr/sbin/nologin overpass
sudo usermod -aG overpass "$USER"
mkdir -p "$OVERPASS_DB_DIR" "$OVERPASS_BACKUP_DIR"
chown -R overpass:overpass "$OVERPASS_DB_DIR" "$OVERPASS_BACKUP_DIR"
sudo chmod g+w "$OVERPASS_DB_DIR" "$OVERPASS_BACKUP_DIR"

Mounting other host directories in the container is optional. See the Directory Structure section below.

⁠Use an Existing Database

If you're upgrading a previous Overpass instance to use the source code or container image from this fork, you can reuse your existing database files. Skip to the Running Overpass section below to start the processes with your existing data.

⁠Initialize the Database from a Clone

Roland Olbricht maintains a daily clone of the full planet data for OpenStreetMap using minutely replication. If you built Overpass directly from source, download the clone using:

OVERPASS_BIN_DIR=   # path to the bin directory in your Overpass installation
OVERPASS_DB_DIR=    # path to your Overpass database directory
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
nohup "$OVERPASS_BIN_DIR/download_clone.sh" \
  --source=http://dev.overpass-api.de/api_drolbr/ \
  --db-dir="$OVERPASS_DB_DIR" \
  --meta="$OVERPASS_META_MODE" &

Or using the container image:

OVERPASS_DB_DIR=    # path to your Overpass database directory on the host
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
docker run -d \
  -v "$OVERPASS_DB_DIR":/opt/overpass/db \
  --entrypoint /opt/overpass/bin/download_clone.sh \
  --no-healthcheck \
  b1tw153/overpass-api \
  --source="http://dev.overpass-api.de/api_drolbr/" \
  --meta="$OVERPASS_META_MODE"

The database files are large and the download may take some time, so it's best to run it as a background process that will not terminate if the terminal connection is closed. Downloads can fail due to transient network issues. However, the download process can be rerun to resume the download.

⁠Initialize the Database from a Planet File

Importing a planet file gives you complete control over the Overpass database and ensures that it starts from a clean data set. The full planet file is very large. Check the current planet-latest.osm.bz2 file size on planet.openstreetmap.org⁠ and make sure you have plenty of disk space for both the planet file and the Overpass database files. Downloading and importing a full planet file can take a couple of days. Make sure your system can run this task in the background.

The import_osm_data.sh script takes care of downloading the planet file, verifying the MD5 checksum, importing the data, and setting the initial replicate_id file based on a chosen replication source. The script supports both HTTP and BitTorrent downloads (if you have aria2c installed), and will import either .osm.bz2 or .pbf files (if you have osmium installed).

If you built Overpass directly from the source code:

PLANET_FILE_URL=    # URL of the planet file to import
OVERPASS_DIFF_URL=  # URL of the chosen replication source associated with the planet file
OVERPASS_BIN_DIR=   # path to the bin directory in your Overpass installation
OVERPASS_DB_DIR=    # path to your Overpass database directory
OVERPASS_DIFF_DIR=  # path to the directory that will be used to store diff files
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
nohup "$OVERPASS_BIN_DIR/import_osm_data.sh" \
  --db-dir="$OVERPASS_DB_DIR" \
  --diff-dir="$OVERPASS_DIFF_DIR" \
  --diff-url="$OVERPASS_DIFF_URL" \
  --data-source="$PLANET_FILE_URL" \
  --meta="$OVERPASS_META_MODE" &

The container image already has aria2c and osmium built in.

PLANET_FILE_URL=    # URL of the planet file to import
OVERPASS_DIFF_URL=  # URL of the chosen replication source associated with the planet file
OVERPASS_DB_DIR=    # path to your Overpass database directory
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
docker run -d --rm \
  -v "$OVERPASS_DB_DIR":/opt/overpass/db \
  --entrypoint /opt/overpass/bin/import_osm_data.sh \
  --no-healthcheck \
  b1tw153/overpass-api \
  --diff-url="$OVERPASS_DIFF_URL" \
  --data-source="$PLANET_FILE_URL" \
  --meta="$OVERPASS_META_MODE"

⁠Initialize the Database from an Extract

Importing an extract allows you to work with a slice of the global data set, which uses fewer resources and can make query responses faster. There are several sources for extracts⁠ which vary in the regions they cover, the frequency of updates, the availability of diff files (critical for Overpass), and the metadata that is included.

Once you've chosen an extract with diff files, initializing the database is similar to the full planet download but with a smaller file.

If you built Overpass directly from source code:

EXTRACT_FILE_URL=   # URL of the extract file to import
OVERPASS_DIFF_URL=  # URL of the chosen replication source associated with the extract file
OVERPASS_BIN_DIR=   # path to the bin directory in your Overpass installation
OVERPASS_DB_DIR=    # path to your Overpass database directory
OVERPASS_DIFF_DIR=  # path to the directory that will be used to store diff files
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
nohup "$OVERPASS_BIN_DIR/import_osm_data.sh" \
  --db-dir="$OVERPASS_DB_DIR" \
  --diff-dir="$OVERPASS_DIFF_DIR" \
  --diff-url="$OVERPASS_DIFF_URL" \
  --data-source="$EXTRACT_FILE_URL" \
  --meta="$OVERPASS_META_MODE" &

Or if you're using the container image:

EXTRACT_FILE_URL=   # URL of the extract file to import
OVERPASS_DIFF_URL=  # URL of the chosen replication source associated with the extract file
OVERPASS_DB_DIR=    # path to your Overpass database directory
OVERPASS_DIFF_DIR=  # path to the directory that will be used to store diff files
OVERPASS_META_MODE= # yes|no|attic - include meta data, base data only, or attic data
docker run -d --rm \
  -v "$OVERPASS_DB_DIR":/opt/overpass/db \
  --entrypoint /opt/overpass/bin/import_osm_data.sh \
  --no-healthcheck \
  b1tw153/overpass-api \
  --diff-url="$OVERPASS_DIFF_URL" \
  --data-source="$EXTRACT_FILE_URL" \
  --meta="$OVERPASS_META_MODE"

⁠Running Overpass

After you have downloaded a database clone or imported a planet file or extract, or if you have an existing database, Overpass is ready to run.

If you're using the run_osm3s.sh script, it automatically detects the update interval for your replication source, and the scripts adapt to that interval with pre-configured timing.

If you built Overpass from the source code:

OVERPASS_BIN_DIR=                 # path to the bin directory in your Overpass installation
export OVERPASS_REPLICATE_ID=auto # use the replicate_id file from the database directory
export OVERPASS_DB_DIR=           # path to your Overpass database directory
export OVERPASS_DIFF_DIR=         # path to the directory that will be used to store diff files
export OVERPASS_DIFF_URL=         # URL of the replication source that matches the database
export OVERPASS_UPDATE_FREQUENCY= # update interval in seconds (should match replication source)
export OVERPASS_META_MODE=        # yes|no|attic - include meta data, base data only, or attic data (should match existing database or import)
export OVERPASS_AREAS=            # yes|no - create or skip derived area data
nohup "$OVERPASS_BIN_DIR/run_osm3s.sh" &

And if you built from the source code, you will need to run your own web server with the /api path mapped to the Overpass cgi-bin directory. See the various guides at the top of this README for information on how to set that up.

Or if you're using the container image, it comes preconfigured with nginx:

OVERPASS_DB_DIR=                  # path to your Overpass database directory
export OVERPASS_REPLICATE_ID=auto # use the replicate_id file from the database directory
export OVERPASS_DIFF_URL=         # URL of the replication source that matches the database
export OVERPASS_UPDATE_FREQUENCY= # update interval in seconds (should match replication source)
export OVERPASS_META_MODE=        # yes|no|attic - include meta data, base data only, or attic data (should match existing database or import)
export OVERPASS_AREAS=            # yes|no - create or skip derived area data
docker run -d \
  -v "$OVERPASS_DB_DIR":/opt/overpass/db \
  -e OVERPASS_REPLICATE_ID \
  -e OVERPASS_DIFF_URL \
  -e OVERPASS_UPDATE_FREQUENCY \
  -e OVERPASS_META_MODE \
  -e OVERPASS_AREAS \
  -p 80:8080 \
  b1tw153/overpass-api

Alternatively, you can set the container parameters as environment variables using a .env script or your preferred container orchestration environment. Mount the OVERPASS_DB_DIR and OVERPASS_DIFF_DIR directories to the predefined paths, keep the variables local and leave them unset in the container.

⁠Area Generation at First Startup

When Overpass starts from a database clone, extract file, or planet file (or if the database is restored from a backup), the database will be significantly behind the current replication state and it will take some time for the database to catch up.

Also at first startup, the area files will be missing from the database. If you're running Overpass in the container or starting it with run_osm3s.sh and area generation is enabled, the default behavior is to generate area files immediately. That can take some time, but the processing is not typically an issue. However, if the host is resource constrained, area generation can slow down the process of catching up to the initial replication state.

If AREA_UPDATE_TIME is also set (see overpass.env), the initial area generation at first startup will be deferred until the specified time.

On resource constrained hosts, it may be better to run Overpass without area generation, to defer area generation until a specified time using AREA_UPDATE_TIME, or to manually control area generation by starting Overpass without area generation while replication catches up then restarting with area generation.

⁠Maintenance

⁠Monitoring Overpass

The output from run_osm3s.sh will give you the status of the Overpass executables. There are several executables to watch for:

• The base dispatcher (dispatcher --osm-base), which controls access to the base, meta, and attic database files
• The areas dispatcher (dispatcher --areas), which controls access to the area database files
• fetch_osc.sh, which downloads diff files from the replication source
• apply_osc_to_db.sh, which unzips the .osc files from the replication source and sends them to update_database
• update_database, which writes the changes from .osc files to the database (runs only during database updates)
• rules_loop.sh, which periodically invokes the query to regenerate area data (optional)
• osm3s_query, which uses a rules file in the database directory to regenerate area data (runs only during area updates)
• backup.sh, which periodically copies the database files to a backup directory (optional)

If any of the core executables stops running, run_osm3s.sh will attempt to cleanly shut down the rest of the system.

The safest way to shutdown Overpass is to send SIGTERM to run_osm3s.sh. On bare metal, that's kill "$RUN_OSM3S_PID". With the container image, that's docker stop. In either case, the run_osm3s.sh script will attempt to stop the components without interrupting the update_database process.

Use caution when shutting down Overpass. There is no safe way to interrupt the process of writing to the database. Killing the update_database process while it is running will often result in a corrupted database that must be replaced by restoring the files from a backup, downloading a new clone, or importing a new extract or planet file.

⁠Directory Structure

The Overpass directory structure includes several directories that are populated during the build process, and additional optional or conventional directories.

If you're building and running Overpass directly from source, most of the non-container directories could be wherever you've installed Overpass. However, the backup directory should be on a separate storage device. The "conventional" directories are typically placed in the same Overpass directory, but can be renamed or moved elsewhere with parameter changes.

Inside the container, all these directories reside under /opt/overpass. The only directories that must be mounted from the host are db and backup, because the data in these directories should be retained. Mounting any of the other data directories from the host is optional. You could mount /opt/overpass/diff if you'd like easier access to the diff files or the fetch_osc.log file. And you could mount /opt/overpass/log to get easier access to the nginx logs. But there is little reason to mount run or tmp directories since this information is not meaningful outside of the container.

⁠Log Files

Each of the executables produces log files and/or output files that have status information and may have an explanation if a component has failed:

You can also tail these files to confirm the health