tramsauer/raddo repository overview

⁠raddo: A Python Package for RADOLAN Weather Radar Data Provision

raddo helps you find, download, sort and preprocess RADOLAN weather radar precipitation data for further usage.

• Installation⁠
  • GDAL⁠
  • Direct Install⁠
• Usage⁠
  • CLI Example⁠
  • Crontab⁠
  • Python Script⁠
  • Docker⁠
  • Warnings⁠
• Contributing⁠
• License⁠
• Changelog⁠
• Further Development⁠
• See also⁠

raddo downloads and processes RADOLAN weather radar ASCII data. Downloaded files are sorted in folders based on year and month and may also be decompressed. As next step raddo creates GeoTiffs in generic WGS84 lat/lon coordinates and/or a single NetCDF file upon user request. In case the data is only needed for a smaller region, masking via a shapefile is also possible. For all possibilities on data retrieval and processing see the usage section.

raddo tries to download all recent RADOLAN ASCII files / archives from the DWD FTP server to the specified directory if files do not exist already. A list of dates possibly available (default <current year>-01-01 until today) is used to compare hypothetical available data sets with actual local available ones. So file listing on the FTP side is skipped due to (formerly) unreliable connection.

RADOLAN data from the German Weather Service (Deutscher Wetterdienst, DWD) is copyrighted! Please find the copyright text here⁠. The freely accessible data may be re-used without any restrictions provided that the source reference is indicated, as laid down in the GeoNutzV ordinance.

The RADOLAN precipitation data files are updated daily by DWD.

The data can be found at opendata.dwd.de⁠.

⁠Installation

The software is developed and tested for usage in Linux. The preferred way of installing is in a conda environment because a working GDAL install is more likely with this option. A conda package for raddo will potentially be available in the future through conda forge. However, also direct installation is possible. A pip package is however not provided for that reason. Testing is done using the conda version of GDAL with pytest.

There is also a docker image available at the docker hub⁠ if you don't mind the overhead. See the Docker⁠ section below for instructions.

⁠GDAL

GDAL is a requirement of raddo. Installation of this dependency can be a problem. If errors arise, GDAL binaries might be missing. When using conda, conda install -c conda-forge gdal should work. On Ubuntu (and derivates) using the UbuntuGIS-ppa seems to be working quite well.

⁠Direct Install

Better have GDAL python bindings already installed (see above). Clone this repository, change into new directory and run:

git clone <repo-url>
cd raddo
pip install .

or

pip install -e .

if you want to work on the code.

⁠Usage

Download RADOLAN data from 14 days ago till yesterday to current directory with raddo.

For further arguments consult the help text:

usage: raddo [-h] [-s START] [-e END] [-d DIRECTORY] [-C] [-f] [-x] [-g] [-n]
             [-N OUTFILE] [-m MASK] [-b BUFFERSIZE] [-F] [-D] [-y] [-v]
             [-u URL] [-r ERRORS] [-t]

raddo - utility to download and preprocess RADOLAN RW
        weather radar data from DWD servers.

optional arguments:
  -h, --help            show this help message and exit
  -d DIRECTORY, --directory DIRECTORY
                        Path to local directory where RADOLAN shouldbe (and
                        may already be) saved. Checks for existing files only
                        if this flag is set. Default: current directory.
  -s START, --start START
                        Start date as parsable string (e.g. "2018-05-20").
                        Default: 14 days ago.
  -e END, --end END     End date as parsable string (e.g. "2020-05-20").
                        Default: yesterday
  -C, --complete        Run all subcommands. Same as using flags -fxgn.
  -f, --sort-in-folders
                        Should the data be sorted in folders?
  -x, --extract         Should the data be extracted?
  -g, --geotiff         Set if GeoTiffs in EPSG:4326 should be created for
                        newly downloaded files.
  -n, --netcdf          Create a NetCDF from GeoTiffs?
  -N OUTFILE, --netcdf-file OUTFILE
                        Name of the output NetCDF file.
  -m MASK, --mask MASK  Use mask when creating NetCDF.
  -b BUFFERSIZE, --buffer BUFFERSIZE
                        Buffer in meter around mask shapefile (Default 1400m).
  -F, --force           Forces local file search. Omits faster check of
                        ".raddo_local_files.txt".
  -D, --force-download  Forces download of all files.
  -y, --yes             Skip user input. Just accept to download to current
                        directory if not specified otherwise.
  -v, --version         Print information on software version.
  -u URL, --radolan_server_url URL
                        Path to recent .asc RADOLAN data on DWD servers.
                        Default: https://opendata.dwd.de/climate_environment/C
                        DC/grids_germany/hourly/radolan/recent/asc/
  -r ERRORS, --errors-allowed ERRORS
                        Errors allowed when contacting DWD Server. Default: 5
  -t, --no-time-correction
                        Omit time adjustment to previous hour in netCDF file
                        creation and just use RADOLANs sum up time HH:50
                        (Default: false).

⁠CLI Example

Download data since June 15th 2020 to current directory and sort, extract, create Geotiffs and a NetCDF file:

raddo -s "2020-07-15" -C

Download RADOLAN data to folder1 (-d) from 2020-07-15 (-s) until yesterday (default) for point in shapefile test_pt.shp (-m). Sort and extract nested archives and create GeoTiffs and a single NetCDF file from there (-C). Don't check for available files but just download all needed files (-D):

raddo -d "folder1" -s "2020-07-15" -CD -m "test_pt.shp"

More visual:

⁠Python Script

import raddo as rd

rd.radolan_down(rad_dir_dwd = ...,  )

⁠Docker

Docker lets you run raddo in a containerized form. All depenencies are set up - including GDAL. docker pull tramsauer/raddo gets you the prebuilt image from docker-hub. Alternatively, with the included Dockerfile the image can also be directly built with docker build -t raddo . from the root directory of the repository.

raddo then can be used like this:

docker run -ti --rm -v /tmp/RADOLAN:/data raddo -C -s "20210422"

• -ti: docker runs in an interactive tty
• --rm: the container is destroyed after usage
• -v /tmp/RADOLAN:/data: an existing folder (/tmp/RADOLAN) is connected to the container (internal folder /data)
  • If asked accept to save the data in /data
• raddo: image name, that automatically starts the raddo program
• -C -s ....: after the image name, additional arguments for raddo can be added, here:
  • -C: complete processing
  • -s "20210422": starting date

The data can then be found in the linked folder, e.g. /tmp/RADOLAN.

⁠Warnings

• currently, if a shapefile mask is used, sub-optimal nearest neighbour resampling is applied in the GeoTiff conversion (as other methods were not functional in gdal python bindings..(?)).
• if GeoTiffs are not wanted, they need to be created anyways, and processing might fill up your tempfs in /tmp..
• if multiple polygons are used as mask, they are dissolved & buffered.
• raddo does not recreate nor warn if GeoTiffs are already available.

⁠Contributing

See CONTRIBUTING⁠ document.

⁠License

Please find the license agreement in LICENSE.txt⁠

⁠Changelog

See Changelog⁠ document.

⁠Further Development

• add DOI
• add conda install

⁠See also

• wradlib⁠:

  An Open Source Library for Weather Radar Data Processing

• radproc⁠:

  A GIS-compatible Python-Package for automated RADOLAN Composite Processing and Analysis