SeaweedFS with peers detection in swarm for distributed master/filer
10M+
Use as a command -peers=SERVICE_NAME:PORT
The image gets the IP from the peers service name
If no ip is found in peers ( like if there is no peers argument, for example ), uses the -ip=SERVICE_NAME as a fallback
Note: works well with docker volume seaweedfs plugin: https://hub.docker.com/repository/docker/gasparekatapy/seaweedfs_plugin
Version 3.5.0 adds a remotesync.sh to be used for cloud storage 2-way sync. It expects a config.conf file with each line a mounted directory to sync
Version 3.4.3 adds a clean_log script to be used with swarm cronjob, to mitigate topics logs bloating by keeping only WEED_KEEP_DAYS days
Version 3.2.7 adds support for global filer and masters, and detection and removal at start of non reachable masters to keep the raft quorum, even in case of master node migration
Version 3.1.2 adds USE_HOSTNAME: "true" to change -ip= parameter to the local hostname, useful with filer
Version 3.1.1 adds USE_DISCOVER: "true" also for DETECT_MASTERS: "true" with assumption master hostnames is the value of -master o -mserver with added _TASKSLOT, i.e. master_1,master_2,master_3. Docker 20.10.0 auto creates alias for hostname if hostname != container
Version 3.1.0 Adds a discovery service, to be used in combination with envs DETECT_PEERS: "true", USE_DISCOVER: "true" and using as hostname hostname: "master_{{.Task.Slot}}" then the -ip and -peers are rebuild using the container hostnames. This is to try to keep the connections with same hostnames and overcame a limitation with master raft cluster when ips change due to docker redeploy
Version 3.0.0 Updates to latest seaweed code and adds DETECT_MASTERS: "true" to expand -master or -mserver to the internal IP list. DETECT_PEERS: "true" allows also to find the current container ip from the -ip=SERVICE_NAME:PORT command, bypassing the service name VIP, allowing SeaweedFS to handle correctly the connections like multiple filers
Version 2.7.4 Detecs from the -ip parameter the internal ip, and uses PUBLIC_URL as fallback. Fixes a "host unreachable" error when trying to connect from the container to the public ip derived from PUBLIC_URL on the same host as the container ( because of docker iptables isolation rules )
Version 2.6.1 Uses PUBLIC_URL for the -ip parameter. This avoids bad ip detection
Version 2.5.0 Uses the new parameter from upstream for weed mount -volumeServerAccess
Version 2.4.1 Added DETECT_PEERS to bypass DNS resolver for peers. Default is true for backward compatibility
Version 2.4.0 adds large version with :large tags
Version 2.3.0 adds support for volume publicUrl, leveraging Swarm {{.Node.Hostname}} template. If the server has a FQDN hostname, this enhancement allows to use weed mount -outsideContainerClusterMode with master, filer and volumes deployed in swarm
Example
version: '3.9'
services:
master:
image: gasparekatapy/seaweedfs
environment:
DETECT_MASTERS: "false"
DETECT_PEERS: "true"
USE_DISCOVER: "true"
USE_HOSTNAME: "false"
CLUSTER_SIZE: 3
networks:
- net
hostname: "master_{{.Task.Slot}}"
command:
- 'master'
- '-ip=master'
- '-ip.bind=0.0.0.0'
- '-port=9333'
- '-port.grpc=19333'
- '-peers=master:9333'
- '-resumeState=true'
- '-raftHashicorp=true'
- '-raftBootstrap=false'
deploy:
mode: global
placement:
constraints:
- node.role == manager
update_config:
parallelism: 1
failure_action: rollback
order: stop-first
filer:
image: gasparekatapy/seaweedfs
environment:
DETECT_MASTERS: "true"
DETECT_PEERS: "false"
USE_DISCOVER: "true"
USE_HOSTNAME: "true"
CLUSTER_SIZE: 3
ports:
- "8888:8888"
- "18888:18888"
networks:
- net
hostname: "filer_{{.Task.Slot}}"
command:
- 'filer'
- '-ip=filer'
- '-ip.bind=0.0.0.0'
- '-port=8888'
- '-port.grpc=18888'
- '-port.readonly=28888'
- '-master=master:9333'
deploy:
mode: replicated
replicas: 3
placement:
max_replicas_per_node: 1
update_config:
parallelism: 1
failure_action: rollback
order: stop-first
volume:
image: gasparekatapy/seaweedfs
environment:
DETECT_MASTERS: "true"
DETECT_PEERS: "false"
USE_DISCOVER: "true"
USE_HOSTNAME: "true"
CLUSTER_SIZE: 3
PUBLIC_URL: "{{.Node.Hostname}}:8080"
ports:
- target: 8080
published: 8080
protocol: tcp
mode: host
- target: 18080
published: 18080
protocol: tcp
mode: host
hostname: "volume_{{.Task.Slot}}"
networks:
- net
command:
- 'volume'
- '-ip=volume'
- '-ip.bind=0.0.0.0'
- '-port=8080'
- '-port.grpc=18080'
- '-mserver=master:9333'
deploy:
mode: global
update_config:
parallelism: 1
failure_action: rollback
order: stop-first
clean_log:
image: gasparekatapy/seaweedfs
environment:
WEED_MASTER: "master:9333"
WEED_FILER: "filer:8888"
WEED_KEEP_DAYS: 2
networks:
- net
entrypoint: ["/clean_log.sh"]
deploy:
mode: replicated
replicas: 0
restart_policy:
condition: none
labels:
# Swarm cronjob
- swarm.cronjob.enable=true
- swarm.cronjob.schedule=0 0 */3 * *
- swarm.cronjob.skip-running=true
remote2local:
image: gasparekatapy/seaweedfs
environment:
WEED_SYNC_TYPE: "remote"
WEED_MASTER: "master:9333"
WEED_FILER: "filer:8888"
volumes:
- /path/to/config.conf:/config.conf
networks:
- net
entrypoint: ["/remotesync.sh", "/config.conf"]
deploy:
mode: replicated
replicas: 0
restart_policy:
condition: none
labels:
# Swarm cronjob
- swarm.cronjob.enable=true
- swarm.cronjob.schedule=0 * * * *
- swarm.cronjob.skip-running=true
local2remote:
image: gasparekatapy/seaweedfs
environment:
WEED_SYNC_TYPE: "local"
WEED_MASTER: "master:9333"
WEED_FILER: "filer:8888"
volumes:
- /path/to/config.conf:/config.conf
networks:
- net
entrypoint: ["/remotesync.sh", "/config.conf"]
deploy:
mode: replicated
replicas: 1
placement:
max_replicas_per_node: 1
preferences:
- spread: node.labels.datacenter
Works also for -filer.peers and -master.peers for server mode.
NOTE
Version 2.0.0+ is based on a custom build of source code and dockerfile to fix WebDAV error for small files ( < 512 Bytes ) with no mime type.
This version does not install supercronic in the image
Content type
Image
Digest
sha256:d6eff85ed…
Size
46.2 MB
Last updated
about 2 years ago
docker pull gasparekatapy/seaweedfs