This program will download the files and metadata for an organisation's packages from a CKAN implementation.
It puts the information in a subfolder for each package that can be edited and saved back to the parent folder to update the files/metadata for the package.
This can be run as either as a stand-alone python program or as a docker image. For specific instructions see below.
Whether you use the docker or python implementation you'll have to work with the following config parameters:
site - REQUIRED
This is the base URL of the CKAN site you'll be working with.
apikey - OPTIONAL
This is the API key from the CKAN site that authorises you to perform certain actions. It isn't required, but without it you'll only be able to perform public actions on public content.
user_agent - OPTIONAL
If you want to set a custom user agent to be identified by the CKAN site you can do it here.
target_directory - REQUIRED FOR PYTHON
For the python install you'll need to supply a directory for the files to be synchd with.
organisation - REQUIRED
This is the organisation you want to sync the data for.
Python 3 required.
Once Python 3 is installed you will need to install the requirements. These are in the file requitements.txt and can be run with:
pip install -r requirements.txt
Rename settings.cfg.example to settings.cfg and change values as necessary, leave it in the same directory at the program.
The program can be run using:
While you could build the image, it's easiest to pull the image from dockerhub using:
docker pull thompsoj/ckan-watcher
The image can then be run as follows:
sudo docker run -d -v /path/to/your/sync/folder:/data -e ORGANISATION=yourOrganisation -e APIKEY=yourAPIkey thompsoj/ckan-watcher
Please note that the flag "-v /path/to/your/sync/folder:/data" will tell it where to sync your files. Leave the "/data" bit alone.
The full list of environment variables that can be set are in the Dockerfile.
Once started, the program synchs the organisation's files and metadata in te following structure:
/target_directory | v /package | v /backup | v data files metadata
To update the metadata or data, modify files in backup directory, copy to the main package directory (the parent of the backup directory), and the the program will update metadata.