Public | Automated Build

Last pushed: 2 years ago
Short Description
Short description is empty for this repo.
Full Description

s3-sync

Syncs multiple paths from S3 with the local container (or host datavolume)

Variables

  • VERBOSE - enable more logging if set to 1
  • FREQUENCY - How often to sync from S3 (in seconds). Default is 300 seconds.

Config format

The tool to sync the S3 paths reads a config file containing the S3 folder and the local path to sync it to. The format is as follows:

( [s3]=S3BUCKET/folder [local]=/my/local/path1 )
( [s3]=S3BUCKET/folder2 [local]=/my/local/path2 )

Each S3/local path pair should be on a seperate line.

Example Docker runs

On AWS (uses role credentials)

This example mounts the file paths.dat on the docker host into /paths.dat in the container and then passes that as the argument to the s3 sync tool.

docker run -d -e "FREQUENCY=600"            \
        -e "VERBOSE=1"                \
        -v mylocaldir/paths.dat:/paths.dat    \
        signiant/s3-sync            \
        /paths.dat

Outside AWS (needs access/secret key)

This example mounts the file paths.dat on the docker host into /paths.dat in the container and then passes that as the argument to the s3 sync tool.

docker run -d     -e "FREQUENCY=600"            \
        -e "VERBOSE=1"                \
        -e "AWS_ACCESS_KEY_ID=MY_ACCESS_KEY    \
        -e "AWS_SECRET_ACCESS_KEY=MY_SECRET    \
        -v mylocaldir/paths.dat:/paths.dat    \
        signiant/s3-sync            \
        /paths.dat
Docker Pull Command
Owner
signiant
Source Repository