Public Repository

Last pushed: 2 months ago
Short Description
Demo with WPS Processes for Copernicus
Full Description


.. image::
:alt: Travis Build

Demo with WPS processes for copernicus.


In Copernicus we want to provide processing capabilities next to a climate data store.
The processing capabilities are exposed using the Web Processing Service standard interface with
the PyWPS
implementation. This demo contains several example processes implemented with
ESMValTool and NCL. The processes are defined with different WPS profiles
mainly to show ways how the input data of the tools can be described and restricted.


The installation is done with Buildout. It is using the Python distribution
system Anaconda
to maintain software dependencies.

If Anaconda is not available then a minimal Anaconda will be installed during
the installation process in your home directory ~/anaconda.

The installation process setups a conda environment named copernicus. All
additional packages are going into this conda environment.
The location is ~/.conda/envs/copernicus.

Now, check out the code from the GitHub repo and start the installation::

$ git clone
$ cd copernicus-wps-demo
$ make clean install

After successful installation you need to start the services. All installed files (config etc ...) are by default in your home directory ~/birdhouse. Now, start the services::

$ make start # starts supervisor services
$ make status # shows supervisor status

The deployed WPS service is available on http://localhost:8096/wps?service=WPS&version=1.0.0&request=GetCapabilities.

Check the log files for errors::

$ cd ~/birdhouse
$ tail -f var/log/pywps/copernicus.log
$ tail -f var/log/supervisor/copernicus.log

For other install options run make help and read the documention of the
Makefile <>_.


If you want to run on a different hostname or port then change the default values in custom.cfg::

$ cd wps
$ vim custom.cfg
$ cat custom.cfg
hostname = localhost
http-port = 8096
archive-root = /path/to/esgf/cmip5/archive

The path to ESGF archive is configured in custom.cfg with the archive-root option.
The configuration file esgf_config.xml for the ESGF coupling module will be generated.

After any change to your custom.cfg you need to run make update (offline mode) or make install again
and restart the supervisor service::

$ make install
$ make restart
$ make status

Using Docker

Get docker images using docker-compose::

$ docker-compose pull

Start the demo with docker-compose::

$ docker-compose up -d  # runs with -d in the background
$ docker-compose logs -f  # check the logs if running in background

By default the WPS service should be available on port 8080::

$ firefox "http://localhost:8080/wps?service=wps&request=GetCapabilities"

Alternatively you can change the port by using environment variables, for example::

$ HTTP_PORT=8096 docker-compose up  # wps service will be available on port 8096

Use docker-compose to stop the containers::

$ docker-compose down


For the demo processes you can fetch CMIP5 test-data from the ESGF archive.
You need a valid ESGF credentials which you can fetch for example with esgf-pyclient_.

  • Tutorial Diagnostic MyDiag

ESGF search facets::

project=CMIP5, experiment=historical, ensemble=r1i1p1, variable=ta, model=MPI-ESM-LR, time_frequency=mon
  • Surface Contour Plot for Precipitation

ESGF search facets::

project=CMIP5, experiment=historical, ensemble=r1i1p1, variable=pr, model=MPI-ESM-LR, time_frequency=mon
  • Timeservices Plot

ESGF search facets::

project=CMIP5, experiment=historical, ensemble=r1i1p1, variable=tas, model=MPI-ESM-LR, time_frequency=mon

You can use wget to download ESGF NetCDF files (-x option to create directories)::

$ wget --certificate cert.pem --private-key cert.pem --ca-certificate cert.pem -N -x -P /path/to/esgf/cmip5/archive

.. _Copernicus:
.. _PyWPS:
.. _ESMValTool:
.. _NCL:
.. _esgf-pyclient:
.. _Buildout:
.. _Anaconda:

Docker Pull Command