This is an image to use aws s3 copy command as a Bitbucket pipe
1.1K
This pipe is a way to execute the copy command to S3 using the aws-cli official pachage from Amazon Web Services
Add the following snippet to the script section of your bitbucket-pipelines.yml file:
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
AWS_ACCESS_KEY_ID: "<string>"
AWS_SECRET_ACCESS_KEY: "<string>"
AWS_DEFAULT_REGION: "<string>"
SOURCE_PATH: "<string>"
DESTINATION_PATH: "<string>"
DESTINATION_PATH: "<string>"
| Variable | Usage |
|---|---|
| AWS_ACCESS_KEY_ID (**) | The IAM Role Access key ID generated in the AWS Console. You can see Access Key and Secret Access Key for more information |
| AWS_SECRET_ACCESS_KEY (**) | The IAM Role Access key generated in the AWS Console. You can see Access Key and Secret Access Key for more information |
| AWS_DEFAULT_REGION (**) | The AWS region code who containing the AWS resources. You can see AWS service endpoints for more information |
| SOURCE_PATH (*) | The source path/bucket where is/are the file(s) that we want to copy. Bucket example: "s3://first-bucket-name/path/file.txt". Path example: "dist/file.txt". |
| DESTINATION_PATH (*) | The destination path/bucket where we want copy the file(s). Bucket example: "s3://second-bucket-name/path/file.txt". Path example: "dist/file.txt". |
| CONTENT_ENCODING | Content encodings that have been applied to the object. |
| ACL | ACL for the object when the command is performed. Valid values are: private, public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control. Default: private. |
| STORAGE_CLASS | Type of storage to use for the object. Valid options are STANDARD, REDUCED_REDUNDANCY, STANDARD_IA, ONEZONE_IA. Default: STANDARD. |
| CACHE_CONTROL | Caching behavior along the request/reply chain. Valid options are no-cache, no-store, max-age=<seconds>, s-maxage=<seconds> no-transform, public, private. Default: no-cache. |
| EXPIRES | Date and time at which the object is no longer cacheable. ISO 8601 format: YYYY-MM-DDThh:mm:ssTZD. Defaults to unset. |
| RECURSIVE_FLAG | Command is performed on all files or objects under the specified directory or prefix. Default: false. |
| EXTRA_ARGS | Extra arguments to be passed to the CLI (see AWS docs for more details). Defaults to unset. |
| DEBUG | Turn on extra debug information. Default: false. |
| SHARED_STORAGE_DIR_PATH (Optional) | Specifies the destination path where will be download the content. Use this parameter to download the content at the repository root. Por Bitbucket, E.g: '/opt/atlassian/pipelines/agent/build' or '/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes' if you want to share the content with another Pipe . Default: ".". |
(*) = required variable. (**) = required variable. If this variable is configured as a repository, account or environment variable, it doesn’t need to be declared in the pipe as it will be taken from the context. It can still be overridden when using the pipe.
More info about parameters and values can be found in the AWS official documentation: AWS docs
Download file from S3.
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
SOURCE_PATH: "s3://bucket-name/folder/file.txt"
DESTINATION_PATH: "path/file.txt"
SHARED_STORAGE_DIR_PATH: "/opt/atlassian/pipelines/agent/build" // This line will copy the path/file.txt into the repository root path
Upload file to S3.
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
SOURCE_PATH: "path/file.txt"
DESTINATION_PATH: "s3://bucket-name/folder/file.txt"
Copy file from an S3 Bucket to another.
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: "us-east-1"
SOURCE_PATH: "s3://first-bucket-name/folder/.env"
DESTINATION_PATH: "s3://second-bucket-name/folder/.env"
Copy folder recursivly with ACL to S3.
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: "us-east-1"
SOURCE_PATH: "dist"
DESTINATION_PATH: "s3://bucket-name/"
RECURSIVE_FLAG: "true"
ACL: "public-read"
Download env file from an S3 Bucket and deploy a NodeJs application to another.
script:
- pipe: docker://helixtech/aws-s3-copy:0.1.0
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: "us-east-1"
SOURCE_PATH: "s3://bucket-name/folder/.env"
DESTINATION_PATH: ".env.production"
SHARED_STORAGE_DIR_PATH: $SHARED_STORAGE_DIR_PATH // This line specify the storage to copy the downloaded file(s)
- npm install
- npm run build
- pipe: atlassian/aws-s3-deploy:latest
variables:
AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
S3_BUCKET: "my-bucket-name"
LOCAL_PATH: "dist"
docker run \
-e AWS_ACCESS_KEY_ID=$MY_AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$MY_AWS_SECRET_ACCESS_KEY \
-e AWS_DEFAULT_REGION=us-east-1 \
-e SOURCE_PATH=s3://my-bucket-name/path-to-files/.env \
-e DESTINATION_PATH=.env \
-v $(pwd):/home/files \
-w /home/files \
--name copy_container \
helictech/aws-s3-copy:latest
If you’d like help with this pipe, or you have an issue or feature request, let us know. The pipe is maintained by [email protected].
If you’re reporting an issue, please include:
Content type
Image
Digest
Size
33.1 MB
Last updated
over 6 years ago
docker pull helixtech/aws-s3-copy