Public | Automated Build

Last pushed: a year ago
Short Description
Build giosystem-core image
Full Description

giosystemcore

This project hosts the core code for the GIOSystem processing line packages

Documentation for giosystem is available at
http://giosystemcore.readthedocs.org

Installation

For the installation process we are going to declare the following shell
variables (adjust them to suit your needs):

.. code:: bash

GIT_USER_NAME=ipmagio
GIT_USER_EMAIL=helpdesk.copernicus@ipma.pt
PROJECT_NAME=giosystem
PROJ_HOME=$HOME/operational/code
VIRTUALENVS_ROOT=$HOME/.virtualenvs

Installation of giosystemcore involves the following steps:

#. Installing system-wide requirements

.. code:: bash

  sudo apt-get install build-essential git cgi-mapserver \
      python-mapscript libxml2 libxml2-dev libxslt1.1 \
      libxslt1-dev gdal-bin libgdal-dev python-dev \
      python-gdal python-numpy python-numexpr \
      python-tables python-setuptools python-virtualenv \
      virtualenvwrapper python-pip gfortran

.. note::

  If this is your first time using git, configure it with:

  .. code:: bash

     git config --global user.name $GIT_USER_NAME
     git config --global user.email $GIT_USER_EMAIL

#. Creating the root directory for the code:

.. code:: bash

  mkdir --parents $PROJ_HOME
  cd $PROJ_HOME

#. Setting up virtualenvwrapper, a tool that facilitates the creation of
isolated Python environments

.. code:: bash

  echo "# virtualenvwrapper variables" >> ~/.bashrc
  echo "export WORKON_HOME=$VIRTUALENVS_ROOT" >> ~/.bashrc
  echo "export PROJECT_HOME=$PROJ_HOME" >> ~/.bashrc
  echo "export PIP_DOWNLOAD_CACHE=$HOME/.pip-downloads" >> ~/.bashrc
  echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc
  source ~/.bashrc

#. Creating a virtualenv.

.. code:: bash

  mkproject $PROJECT_NAME

.. note::

  In order to use the virtualenv it has to be activated. Upon creation,
  it is automatically activated. To activate it on some other occasion,
  use:

  .. code:: bash

     workon $PROJECT_NAME

#. linking system-wide python libs to the virtualenv. Some python libraries are
troublesome to install directly into the virtualenv. For these cases we can
link the relevant system libraries inside the virtualenv.

.. code:: bash

  ln --symbolic --force /usr/lib/python2.7/dist-packages/numpy $VIRTUAL_ENV/lib/python2.7/site-packages
  ln --symbolic --force /usr/lib/python2.7/dist-packages/numexpr $VIRTUAL_ENV/lib/python2.7/site-packages
  ln --symbolic --force /usr/lib/python2.7/dist-packages/tables $VIRTUAL_ENV/lib/python2.7/site-packages
  ln --symbolic --force /usr/lib/python2.7/dist-packages/osgeo $VIRTUAL_ENV/lib/python2.7/site-packages
  ln --symbolic --force /usr/lib/python2.7/dist-packages/mapscript.py $VIRTUAL_ENV/lib/python2.7/site-packages
  ln --symbolic --force /usr/lib/python2.7/dist-packages/_mapscript.so $VIRTUAL_ENV/lib/python2.7/site-packages

#. Installing the package. Finally we can install the giosystemcore package.
We download the code from its source code repository and use pip to install
it.

.. code:: bash

  git clone https://$GIT_USER_NAME@bitbucket.org/ipmagio/giosystem-core.git
  pip install --editable giosystem-core
  pip install ipython  #  a nicer python shell

This will install giosystemcore in editable mode, which means that we get
a normal git repository that can be used to work on the code.

Installing giosystemcore product generating algorithms

In order to be able to use the scripts that automate installation of the
external libraries and product generation algorithms you must use the settings
from an already configured instance of the django-giosystem-settings app.

  1. Download, compile and install the HDF5, SZIP, ZLIB libraries into the
    locations defined in the settings by running the install_giosystem_hdf5
    script. This script has been installed by pip when installing
    giosystemcore.

    .. code:: bash

    install_giosystem_hdf5 http://gio-gl.meteo.pt/giosystem/settings/api/v1/

#. Download, compile and install each of the giosystem external packages by
running the install_giosystem_algorithms script

.. code:: bash

  install_giosystem_algorithms http://gio-gl.meteo.pt/giosystem/settings/api/v1/

Tagging new versions of algorithms

Whenever a new version of an external Fortran or C/C++ algorithm is available
to be included in giosystemprocessing, the following procedure must be
performed:

  • Be sure that the stable code is in the current version of the master branch
    of the package's source code repository at bitbucket

  • Edit the 'giosystemcore/giosystemcore/scripts/installalgorithms.py' script in
    order to add/change the respective package repository url and the installation method.

  • Create a git tag, specifying the version number to release

    .. code:: bash

    cd <package_source_code_repository>
    git tag -a v<version_number> -m 'Tagged version <version_number>'
    git push origin v<version_number>

Additional dependencies

Additionally, install the following dependencies inside the virtualenv:

.. code:: bash

  pip install Sphinx nose

The sphinx package is used to work on the documentation of the project. The
nose package is used for automated test managing.

Docker Pull Command
Owner
ipmacopernicus
Source Repository

Comments (0)