Sign inSign up

linuxchina/local-s3

By linuxchina

•Updated 6 days ago

Amazon S3 service implementation based on Netty and GraalVM native image

Image
0

123

linuxchina/local-s3 repository overview

LocalS3 allows a developer to launch an Amazon S3 service in the local development machine without hitting AWS. LocalS3 implements the most commonly used APIs of AmazonS3, including versioning objects, multipart uploads, etc. You can run LocalS3 as a web service or as an external resource for testing. Visit the GitHub repository⁠ for more details about LocalS3.

⁠Features
  • Light and fast: only Java 25, Netty, Jackson 3
  • 90+ S3 operations to be compitable with S3 APIs
  • https/http support
  • Health check and admin endpoints: /admin/_ui
  • A built-in Iceberg REST catalog
  • Static website hosting
⁠Usage

You can map a port, like 29090, in the host machine to container port 29090. Then you can hit the endpoint http://localhost:29090⁠ to perform Amazon S3 operations.

docker run --name s3 -d -v C:\\local-s3:/data -p 29090:29090 linuxchina/local-s3 Here is a Java example that visits LocalS3 with AmazonS3Client. Unlike hitting the real AmazonS3, you need to configure the endpoint and enable path-style access for the AmazonS3Client.

AwsClientBuilder.EndpointConfiguration endpoint = new AwsClientBuilder.EndpointConfiguration("http://localhost:29090", "local");
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
    .enablePathStyleAccess()
    .withEndpointConfiguration(endpoint)
    .build();
s3.createBucket("my-bucket");

There is also a testcontinaers Java implementation to run LocalS3 with testcontainers API.

⁠Tags

Each release has 3 Docker image tags: latest, ${version}, and native-${version}. The latest tag is the same as the ${version} tag; LocalS3 is packaged as a jar file and runs on a stock JDK in this type of tag. The tag with the native- prefix runs a GraalVM Native Image built executable, which is much smaller and starts up faster.

TagsImage sizeDescription
latest or ${version}More than 125MBA JAR file running on a stock JDK
native-${image}Less than 28MBA native executable

Retain your data LocalS3 stores data in the /data path of the Docker container. If you want to retain data after removing the container. You need to mount a volume or directory to that path.

docker run --name s3 -d -v C:\\local-s3:/data -p 8080:80 linuxchina/local-s3 Run LocalS3 in IN_MEMORY mode with initial data LocalS3 loads data from the specified volume or path when starting the container. All changes only apply to the memory and won't be persisted to the volume.

docker run -d -p 29090:29090 -e LOCAL_S3_MODE=IN_MEMORY -v C:\\local-s3:/data linuxchina/local-s3 Environment Variables LOCAL_S3_MODE Valid values are PERSISTENCE(default) and IN_MEMORY. In PERSISTENCE mode, LocalS3 stores all data in the file system. With IN_MEMORY mode, all data exist in the memory, and LocalS3 has the best performance but will wipe everything after stopping the container.

docker run -d -p 29090:29090 -e MODE=IN_MEMORY linuxchina/local-s3 LOCAL_S3_LOGGING_LEVEL Valid values are TRACE, DEBUG, INFO, WARN, and ERROR. This environment variable indicates the logging level of LocalS3 in the Docker container. The default value is INFO.

docker run -d -p 29090:29090 -e LOCAL_S3_LOGGING_LEVEL=WARN linuxchina/local-s3

Tag summary

Content type

Image

Digest

sha256:f3ee5d197…

Size

125.1 MB

Last updated

6 days ago

docker pull linuxchina/local-s3