Ceph Rados Gateway
Run your own distributed s3 storage in docker.
Ceph cluster creation
Let's assume that you want to test ceph on single machine.
Our lan ip is 192.168.0.7
and hostname is callisto
.
To keep configs in sync we will bind-mount them from host
dir /media/backups/ceph/etc
. We will use s3.rgw.ceph
as root domain for s3 communication.
Run monitor node:
docker run -d --net=host -e MON_IP=192.168.0.7 -e MON_NAME=callisto \
-v /media/backups/ceph/etc:/etc/ceph --name=ceph.mon0 ulexus/ceph-mon
To proceed, we need to enter ceph.mon0
(docker-enter or nsenter) and
allocate 3 osd nodes for future use:
ceph osd create
ceph osd create
ceph osd create
Run 3 osd nodes with ids 0, 1 and 2:
docker run -d -e OSD_ID=0 -v /media/backups/ceph/etc:/etc/ceph \
-v /media/backups/ceph/osd.0:/var/lib/ceph/osd/ceph-0 --name ceph.osd0 ulexus/ceph-osd
docker run -d -e OSD_ID=1 -v /media/backups/ceph/etc:/etc/ceph \
-v /media/backups/ceph/osd.1:/var/lib/ceph/osd/ceph-1 --name ceph.osd1 ulexus/ceph-osd
docker run -d -e OSD_ID=2 -v /media/backups/ceph/etc:/etc/ceph \
-v /media/backups/ceph/osd.2:/var/lib/ceph/osd/ceph-2 --name ceph.osd2 ulexus/ceph-osd
In ceph.mon0
run following commands:
ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
chmod +r /etc/ceph/ceph.client.radosgw.keyring
ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.radosgw.gateway --gen-key
ceph-authtool -n client.radosgw.gateway --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.radosgw.gateway -i /etc/ceph/ceph.client.radosgw.keyring
Add following to your /etc/ceph/ceph.conf
:
[client.radosgw.gateway]
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/ceph/client.radosgw.gateway.log
rgw print continue = false
rgw_dns_name = s3.rgw.ceph
Enter ceph.mon0
once again to create user:
apt-get update
apt-get install radosgw
radosgw-admin --name client.radosgw.gateway user create --uid alice --display-name "Alice"
You will get keys to use with s3 in the output. Note that in json output
some chars could be escaped, you don't need extra \
in your s3 client settings.
Now let's run rados gateway at last:
docker run -d -p 192.168.0.7:80:80 -v /media/backups/ceph/etc:/etc/ceph \
--name ceph.radosgw0 bobrik/radosgw
Using s3 storage
Usually s3 clients address buckets in format <bucket>.<s3_host>
.
To create and use buckets we need to point <s3_host>
to 192.168.0.7
with all subdomains. If you don't have dns, just add following to /etc/hosts
to use s3.rgw.ceph
as <s3_host>
:
192.168.0.7 s3.rgw.ceph whatever.s3.rgw.ceph
This will also allow you to use bucket whatever
. If you need more buckets,
just add them to /etc/hosts
.
Release history
bobrik/radosgw:1.0
first versioned release.