Sign inSign up

windpioneers/gdal-python

By windpioneers

Updated 4 months ago

GDAL + Python images for development and production

Image
0

10K+

windpioneers/gdal-python repository overview

Python + Geo Tools - production and development images

Can be used to develop and productionise a geodjango project or other scientific / geospatial services.

A range of tools are included - the most important version changes being gdal and python.

But, these tools and configuration can change and be quite fast moving compared to the underlying stack. So, we use a release name in combination with the key versions of gdal and python to form image tags.

The tag format is:

windpioneers/gdal-python:<release-name>-gdal-<gdal-version>-python-<python-version>-<build-mode>

Examples:

  • windpioneers/gdal-python:button-gdal-2.4.1-python-3.9-dev, or
  • windpioneers/gdal-python:monster-gdal-2.4.1-python-3.8-slim

You'll want to check the release notes for this repository to understand what's in the different named releases.

For development

Images with the -dev tag suffix are based on Microsoft devcontainers, which are useful universally, but especially as a base for .devcontainers and remote code collaboration in VSCode.

These images include additional development features, including:

  • A fully set up zsh terminal with oh-my-zsh, that maps git and SSH to your local machine (seamless git commands and shortcuts from inside your container)
  • Poetry pre-installed
  • Node.js pre-installed with prettier, enabling format-on-save within vscode
  • Austin profiler, allowing flame-graph visualisation of python process memory and CPU requirements for optimisation

For production

Images with the -slim tag suffix are based on python's buster-slim images, providing a minimal installation of OS dependencies for quick and low-memory container orchestration, and low memory usage in serverless environments.

Of course, these images have none of the developer tools installed but do have poetry for installation of your app dependencies.

If you end up needing a development dependency in production (for some reason), that's not a fundamental problem - the -dev images shouldn't introduce any insecurities, but will take longer to pass through build process, and cost more to run day-to-day.

Geo Tools

Included in the geo stack is:

  • libkml (Google's version, which is compatible with various features used in Google Maps and Earth that the default libkml isn't)
  • GDAL (compiled against the Google libkml)
  • GEOS
  • PROJ

Using .devcontainers

Follow VSCode's instructions for using .devcontainers (there are endless online tutorials). You'll want to base your devcontainer on one of the -dev images.

Your .devcontainer/Dockerfile should look like this:


FROM windpioneers/gdal-python:gdal-2.4.1-python-3.9-dev

# Tell zsh where you want to store history

# We leave you to decide, but if you put this into a folder that's been mapped

# into the container, then history will persist over container rebuilds :)

#

# !!!IMPORTANT!!!

# Make sure your .zsh_history file is NOT committed into your repository, as it can contain

# sensitive information. So in this case, you should add

# .devcontainer/.zsh_history

# to your .gitignore file.

#

ENV HISTFILE="$PWD/.devcontainer/.zsh_history"

# Install and cache poetry dependencies

# Why don't we do this for you in the gdal-python image???

# Well, it's because we can't conditionally mount a cache (for dev/slim images).

# Also, if you're not using poetry, it doesn't make sense for us to force a cache.

#

COPY pyproject.toml poetry.lock ./
RUN --mount=type=cache,id=dev-cache,target=/root/.cache \
 poetry config virtualenvs.create false && \
 poetry install $(test "$INSTALL_DEV_TOOLS" != true && echo "--no-dev") --no-interaction --no-ansi

# Setting this ensures print statements and log messages promptly appear

ENV PYTHONUNBUFFERED TRUE

These are highly opinionated settings which you can change for yourself. But we thing your .devcontainer/devcontainer.json should look like this:

// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.187.0/containers/python-3
{
  "name": "GeoPython",
  "build": {
    "dockerfile": "Dockerfile",
    "context": "..",
  },

  // Set *default* container specific settings.json values on container create.
  "settings": {
    "austin.mode": "Wall time",
    "editor.defaultFormatter": "esbenp.prettier-vscode",
    "editor.formatOnSave": true,
    "jupyter.widgetScriptSources": ["jsdelivr.com", "unpkg.com"],
    "prettier.prettierPath": "/usr/local/prettier", // Note: Prettier is installed (with its requisite installation of node to run it) already, and linked here
    "python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
    "python.formatting.blackPath": "/usr/local/py-utils/bin/black",
    "python.formatting.provider": "black",
    "python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
    "python.languageServer": "Pylance",
    "python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
    "python.linting.enabled": true,
    "python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
    "python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
    "python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
    "python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
    // Line length to match black settings
    // Disabling specific messages:
    //  - To find the details do: /usr/local/py-utils/bin/pylint --list-msgs
    //  - Disable missing-module-docstring (C0114) because we don't document modules routinely, just their members
    //  - Disable invalid-name (C0103) because pylint thinks that eg 'x', 'df', 'np' are invalid due to their lengths
    "python.linting.pylintArgs": [
      "--max-line-length=120",
      "--disable=missing-module-docstring,invalid-name"
    ],
    "python.linting.pylintEnabled": true,
    "python.linting.pylintPath": "/usr/local/py-utils/bin/pylint",
    "python.pythonPath": "/usr/local/bin/python",
    "terminal.integrated.defaultProfile.linux": "zsh"
  },

  // Add the IDs of extensions you want installed when the container is created.
  "extensions": [
    "esbenp.prettier-vscode",
    "irongeek.vscode-env",
    "mikestead.dotenv",
    "ms-python.python",
    "ms-python.vscode-pylance",
    "ms-toolsai.jupyter",
    "ms-toolsai.jupyter-renderers",
    "ms-toolsai.jupyter-keymap",
    "ms-vsliveshare.vsliveshare",
    "p403n1x87.austin-vscode", // For profiling
    "ritwickdey.liveserver"  // For serving previews of html documents, e.g. those generated by sphinx
  ],

  // Use 'forwardPorts' to make a list of ports inside the container available locally.
  "forwardPorts": [80, 443, 7045, 7046, 7047, 7048, 7049, 8080],

  // Use 'postCreateCommand' to run commands after the container is created.
  "postCreateCommand": "echo \"Install your requirements here\"",

  // Comment out connect as root instead - but some of the devtools might stop working! More info: https://aka.ms/vscode-remote/containers/non-root.
  "remoteUser": "vscode",

  // Allow ptrace based debuggers (like austin) to work in the container
  // DO NOT add these run arguments to containers that are in any way exposed to external users or the internet
  "runArgs": ["--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined"]
}

Tag summary

Content type

Image

Digest

sha256:2a1012ea0

Size

2.3 GB

Last updated

4 months ago

docker pull windpioneers/gdal-python:boiling-gopher-gdal-3.10.0-python-3.12-dev