Public | Automated Build

Last pushed: 4 days ago
Short Description
A DCAT-AP validator for the Flemish Open Data Portal.
Full Description

#+TITLE: VODAP-DCATAP-VALIDATION

  • Introduction

Scripts to use the [[https://github.com/EmidioStani/dcat-ap_validator][dcat-ap validation rules]] to validate currently
available dcat-ap catalogs (to produce a report of the problems for a
given link).

  • Deployment

** new machine setup
To start a fresh, independent environment one has to setup an
up-to-date ubuntu/debian Linux. This and the subsequent deployment of
the application is automated by the vagrant setup, included. Assuming
[[https://www.vagrantup.com]["Vagrant"]] is installed the following is all that should be needed:

#+BEGIN_EXAMPLE
vagrant up

#+END_EXAMPLE

This will start up virtualbox, install a ubuntu image, and then
installs the first the necessary system tools: docker, git,
... defined in bootstrap.sh. The application dependencies are
collected in the subsequent script: application_bootstrap.sh.

** on an existing machine
On an existing machine the setup starts from the
application_bootstrap.sh script. By running this (as sudo) the
necessary application dependencies are installed.

  • Execution

The commands are initiated through the make command.

  • make startupvirtuoso : starts a local virtuoso
  • make loadCatalog : loads the catalog on disk into virtuoso
  • make vodapreport : creates the report in html based on the content in the virtuoso and the activated rules

Commands to manage the catalog state

- make rmCatalog: removes the latest catalog aggregated file from disk 
- make cleanCatalog: removes the latest catalog from disk
- make createCatalog: create the latest catalog (from the existing downloads, if present)

Commands to manage the set of validation rules

- make VODAPrules     : use the rules for VODAP
- make ISAVODAPrules  : use the rules from ISA adapted to VODAP (SPARQL) case
- make ISArules       : use the rules from ISA (as is)
  • As webservice
    [TODO]

    The validation can be deployed as a webservice. On
    http://localhost/ a simple webform is presented to the user. The
    form expects an URL to the catalog in DCAT-AP form serialized in
    RDF. It supports serializations in ntriples, turtle and RDF/XML.

    The webservice will forward then the user to a directory containing validation reports .

    The validation consists of 2 levels:

    • RDF serialization compliance: is the provided file RDF compliant.
      Typical issues are incorrect URI identifiers containing for example spaces.
    • DCAT-AP validation: is the provided file DCAT-AP(VO) compliant.

** building the webservice
[TODO]: make a development and production docker-compose

The creation of a new webservice to test locally is done as follows:

+ Firstly ensure that the application_bootstrap.sh has been run.
+ (re)build the service using the following

#+BEGIN_EXAMPLE

docker-compose -f docker-compose.yml build

#+END_EXAMPLE

+ stopping and starting the previous build
#+BEGIN_EXAMPLE

docker-compose -f docker-compose.yml down
docker-compose -f docker-compose.yml up -d

#+END_EXAMPLE

** deploying the webservice
[TODO]: make a development and production docker-compose

Deploying the ready made build is as easy as the following

#+BEGIN_EXAMPLE
wget https://raw.githubusercontent.com/tenforce/vodap-dcatap-validation/master/docker-compose.yml
#+END_EXAMPLE

This docker-compose file contains the VODAP default settings 

#+BEGIN_EXAMPLE
docker-compose -f docker-compose.yml up -d
#+END_EXAMPLE



To extend the environment there is a 'docker-compose-dev.yml'
file which will override some of the options used in the 
main 'docker-compose.yml' file. To use the file:

#+BEGIN_EXAMPLE
docker-compose -f docker-compose.yml -f docker-compose-dev.yml <command>
#+END_EXAMPLE

Note: The docker-compose command 'extends' has problems with the
'links' so cannot be used to simplify the above.
Docker Pull Command
Owner
bertvannuffelen

Comments (0)