Public | Automated Build

Last pushed: a year ago
Short Description
ckan watcher
Full Description

Description

This program will download the files and metadata for an organisation's packages from a CKAN implementation.

It puts the information in a subfolder for each package that can be edited and saved back to the parent folder to update the files/metadata for the package.

Setup

This can be run as either as a stand-alone python program or as a docker image. For specific instructions see below.

Whether you use the docker or python implementation you'll have to work with the following config parameters:

site - REQUIRED

This is the base URL of the CKAN site you'll be working with.

apikey - OPTIONAL

This is the API key from the CKAN site that authorises you to perform certain actions. It isn't required, but without it you'll only be able to perform public actions on public content.

user_agent - OPTIONAL

If you want to set a custom user agent to be identified by the CKAN site you can do it here.

target_directory - REQUIRED FOR PYTHON

For the python install you'll need to supply a directory for the files to be synchd with.

organisation - REQUIRED

This is the organisation you want to sync the data for.

Python

Python 3 required.

Once Python 3 is installed you will need to install the requirements. These are in the file requitements.txt and can be run with:

pip install -r requirements.txt

Rename settings.cfg.example to settings.cfg and change values as necessary, leave it in the same directory at the program.

The program can be run using:

python watcher.py

Docker

Docker required

While you could build the image, it's easiest to pull the image from dockerhub using:

docker pull thompsoj/ckan-watcher

The image can then be run as follows:

sudo docker run -d -v /path/to/your/sync/folder:/data -e ORGANISATION=yourOrganisation -e APIKEY=yourAPIkey thompsoj/ckan-watcher

Please note that the flag "-v /path/to/your/sync/folder:/data" will tell it where to sync your files. Leave the "/data" bit alone.

The full list of environment variables that can be set are in the Dockerfile.

Usage

Once started, the program synchs the organisation's files and metadata in te following structure:

/target_directory
    |
    v
    /package
        |
        v
        /backup
            |
            v
            data files
            metadata

To update the metadata or data, modify files in backup directory, copy to the main package directory (the parent of the backup directory), and the the program will update metadata.

Docker Pull Command
Owner
thompsoj
Source Repository

Comments (0)