Sign inSign up

greyhard/aws-s3-sync

By greyhard

•Updated 15 days ago

Cron Backup any files to Yandex Object Storage, AWS S3, RustFS

Image
Integration & delivery
0

1.7K

greyhard/aws-s3-sync repository overview

⁠greyhard/aws-s3-sync

MultiPlatform linux/amd64 linux/arm64

Docker container that periodically syncs a folder to Amazon S3 (or S3-compatible storage like Yandex Cloud) using the AWS Command Line Interface tool⁠ and cron.

Built on Alpine Linux 3.22 with Python 3.12 and AWS CLI.

⁠Usage

⁠With Docker Run
docker run -d [OPTIONS] greyhard/aws-s3-sync
⁠With Docker Compose (Recommended)

Create a .env file with your credentials:

AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key

Create a docker-compose.yml:

version: '3.3'
services:
  s3sync:
    image: greyhard/aws-s3-sync
    restart: always
    environment:
      AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
      AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
      AWS_DEFAULT_REGION: ru-central1
      BUCKET: your-bucket-name
      BUCKET_PATH: /your/path
      CRON_SCHEDULE: "0 */4 * * *"
      PARAMS: "--endpoint-url=https://storage.yandexcloud.net"
    volumes:
      - ./data:/data:ro

Run with: docker-compose up -d

⁠Environment Variables

⁠Required Parameters:
  • AWS_ACCESS_KEY_ID: AWS/S3-compatible access key
  • AWS_SECRET_ACCESS_KEY: AWS/S3-compatible secret key
  • AWS_DEFAULT_REGION: Region of your bucket (e.g., ru-central1 for Yandex Cloud)
  • BUCKET: The name of your bucket
  • Volume mount: /data - mount your local folder to this path (use :ro for read-only)
⁠Optional Parameters:
  • PARAMS: Additional parameters for the sync command (AWS S3 sync docs⁠)
    • Example: --endpoint-url=https://storage.yandexcloud.net for Yandex Cloud
  • BUCKET_PATH: Path in your S3 bucket (must start with /), defaults to /
  • CRON_SCHEDULE: Cron schedule for sync (cron syntax⁠), defaults to 0 1 * * * (daily at 1:00 AM)
  • Command now: Run sync once and exit (no cron scheduling)

⁠Examples

⁠Yandex Cloud Storage (every 4 hours)
docker run -d \
    -e AWS_ACCESS_KEY_ID=YCAJE... \
    -e AWS_SECRET_ACCESS_KEY=YCPw... \
    -e AWS_DEFAULT_REGION=ru-central1 \
    -e BUCKET=mybucket \
    -e BUCKET_PATH=/backups \
    -e CRON_SCHEDULE="0 */4 * * *" \
    -e PARAMS="--endpoint-url=https://storage.yandexcloud.net" \
    -v /path/to/data:/data:ro \
    greyhard/aws-s3-sync
⁠AWS S3 (hourly sync)
docker run -d \
    -e AWS_ACCESS_KEY_ID=AKIA... \
    -e AWS_SECRET_ACCESS_KEY=secret... \
    -e AWS_DEFAULT_REGION=us-east-1 \
    -e BUCKET=mybucket \
    -e BUCKET_PATH=/backups \
    -e CRON_SCHEDULE="0 * * * *" \
    -v /home/user/data:/data:ro \
    greyhard/aws-s3-sync
⁠One-time sync (no cron)
docker run --rm \
    -e AWS_ACCESS_KEY_ID=your_key \
    -e AWS_SECRET_ACCESS_KEY=your_secret \
    -e AWS_DEFAULT_REGION=ru-central1 \
    -e BUCKET=mybucket \
    -e PARAMS="--endpoint-url=https://storage.yandexcloud.net" \
    -v /home/user/data:/data:ro \
    greyhard/aws-s3-sync now

⁠Credits

This container is heavily inspired by istepanov/backup-to-s3⁠.

The main difference is that this container is using Alpine Linux instead of Debian to be more light weight. It also uses a different method of using the AWS CLI tool.

Tag summary

Content type

Image

Digest

sha256:049fd7f89…

Size

67.8 MB

Last updated

15 days ago

docker pull greyhard/aws-s3-sync