Sign inSign up

keitaro/ckan-datapusher

By keitaro

•Updated almost 5 years ago

CKAN datapusher

Image
0

100K+

keitaro/ckan-datapusher repository overview

⁠CKAN Datapusher

DataPusher - Automatically add Data to the CKAN DataStore. Dockerfile location⁠

Looking to run CKAN on Kubernetes? Check out our CKAN Helm Chart⁠!

Run CKAN using docker compose⁠.

⁠Overview

All images are based on Alpine Linux⁠ and include only required extensions to start a CKAN instance. The docker images are built using a multi-stage docker approach in order to produce slim production-grade docker images with the right libraries and configuration. This multi-stage approach allows us to build python binary wheels in the build stages that later on we install in the main stage.

⁠Configuration

The following configuration options are available through setting environment variables when running datapusher:

  • MAX_CONTENT_LENGTH - Sets the maximum size in bytes that datapusher should process. Default: 10485760
  • CHUNK_SIZE - Sets the chunk size in bytes with which datapusher will fetch the resource. Default: 16384
  • CHUNK_INSERT_ROWS - Sets the number of rows that datapusher would chunk up the resource in order to import it in CKAN's datastore. Default: 250
  • DOWNLOAD_TIMEOUT - Sets the time in seconds that datapusher would timeout trying to reach a resource. Default: 30
  • SSL_VERIFY - Enable or disable (boolean) verification of SSL when trying to get resources. Default: True
  • REWRITE_RESOURCES - Enable or disable (boolean) whether datapusher should rewrite resources uploaded to CKAN's filestore. Useful when running with docker, since datapusher takes the CKAN Site URL value for generating the resource URL. Default: False
  • REWRITE_URL - Sets the rewrite URL that datapushed will rewrite resources that are uploaded to CKAN's filestore. Default: http://ckan:5000⁠

Tag summary

Content type

Image

Digest

Size

90.1 MB

Last updated

almost 6 years ago

docker pull keitaro/ckan-datapusher