bres/supload repository overview

.. -- mode: rst --

⁠What is supload?

It's simple script for upload file to cloud storage based on OpenStack Swift API <http://docs.openstack.org/trunk/openstack-object-storage/developer/content/>_.

⁠Feature

• recursive upload
• check files by MD5 hash
• upload only modified files
• expiring files
• exclude files matches pattern
• find and upload only newest files
• limit upload speed
• auto unpacks archive on uploading (if storage supports)

⁠Restrictions

• support only less than 5G file to upload

⁠Installation

For use script need installed utilities curl and file. Install for debian/ubuntu::

$ apt-get install curl file

Get script::

$ wget https://raw.github.com/selectel/supload/master/supload.sh && chmod +x supload.sh
# or
$ curl -Ok https://raw.github.com/selectel/supload/master/supload.sh && chmod +x supload.sh

You may move script to /usr/bin or /usr/local/bin if want::

$ mv supload.sh /usr/local/bin/supload

⁠Usage

By default the script work with Selectel Cloud Storage <http://storage.selectel.ru/>_.

Get help::

$ ./supload.sh

Upload one file, file will be uploaded if not existence in storage or local file is different from file in storage (md5 check)::

$ ./supload.sh -u USERNAME -k USERKEY storage_dir local_file.path

Upload all files in directory recursive (option -r)::

$ ./supload.sh -u USRNAME -k USERKEY -r storage_dir local_dir/

Force upload without checking existing files in storage by md5 (option -M)::

$ ./supload.sh -u USERNAME -k USERKEY -r -M storage_dir local_dir/

If the Storage supports expiring object you can set time when uploaded file must be deleted (option -d). Uploaded file will be deleted after 7 days::

$ ./supload.sh -u USERNAME -k USERKEY -d 7d storage_dir file.doc

Exclude some files for upload::

$ ./supload.sh -u USRNAME -k USERKEY -r -e ".git/*" -e "file_in_root_dir" -e "dir1/*/*.cache" -e "*~" -e "*.bak" storage_dir local_dir/

Enable detect mime-type (by utility file) and set Content-Type for uploading file (option -c)::

$ ./supload.sh -u USERNAME -k USERKEY -c storage_dir localfile

Upload only files modified in the last 7 days (option -m)::

$ ./supload.sh -u USERNAME -k USERKEY -r -m -7 storage_dir local_dir/

Limit upload speed (option -s)::

$ ./supload.sh -u USERNAME -k USERKEY -r -s 2M storage_dir local_dir/

Upload and extract archive (option -z)::

$ ./supload.sh -u USERNAME -k USERKEY -z tar.bz2 storage_dir archive.tar.bz2

⁠Changes

3.0:

• unpack archive
• improve handle error and debug output
• add functional tests

2.7:

• limit upload speed (-s)

2.6:

• added new option for find&filter files by modified time p(-m)
• fixed parser for etag header
• added handle for 401 Unauthorized

2.5:

• disabled detect mime-type and set content-type by default
• add option for enable detect mime-type (-c)

2.4:

• fixed: reauth when access denied

2.3:

• hide password key in cmdline

2.2:

• add option for exclude files (-e)

2.1:

• add support for expiring files (-d)
• util file not necessarily now

2.0:

• ignore case for auth headers
• support MacOsX
• small fixes