kamatama41/embulk-executor-remoteserver repository overview

⁠Remote server executor plugin for Embulk

Embulk executor plugin to run Embulk tasks on remote servers.

⁠Overview

• Plugin type: executor

⁠Notes

• It's still very experimental version, so might change its spec without notification.
• It might have some issues or bugs. I would appreciate it if you use this and give me reports/feedback!

⁠Configuration

• hosts: List of remote servers (hostname or hostname:port, default port is 30001). If not specified, the executor runs as the local mode, which starts an Embulk server on its own process (array of string)
• timeout_seconds: Timeout seconds of the whole execution (integer, default: 3600)
• use_tls: Enable to connect server over TLS (boolean, default: false)
• cert_p12_file: Information of a PKCS12 file used as your client certificate. It would be needed when client authentication is enabled on Embulk server.
  • path: Path of the file (string, required)
  • password: Password of the file (string, required)
• ca_cert_path: Path of a CA certificate file. It would be needed when Embulk server uses a certificate signed by an unknown CA.

⁠Example

exec:
  type: remoteserver
  hosts:
    - embulk-server1.local
    - embulk-server2.local:30002
  timeout_seconds: 86400

⁠Embulk server

The server receives requests from client (Embulk) and run Embulk tasks, then returns results to client. It communicates with clients via TCP 30001 port.

⁠Running Embulk server as a Docker container

The image is hosted by DockerHub⁠.
You can try running Embulk server by the following command.

$ docker run --rm -p 30001:30001 kamatama41/embulk-executor-remoteserver

⁠Configure Embulk server

There are some environment variables to configure the server

• BIND_ADDRESS: Bind address of the server (default: 0.0.0.0)
• PORT: Port number to listen (default: 30001)
• USE_TLS: Try to connect to client via TLS if true (default: false)
• REQUIRE_TLS_CLIENT_AUTH: Require client authentication if true (default: false)
• CERT_P12_PATH, CERT_P12_PASSWORD: Path and password of the PKCS12 file for server certificate. Ignored unless both is set.
• CA_CERT_PATH: Path of the CA certificate file. It would be needed when client authentication is enabled and client certificate is signed by an unknown CA.

⁠Build

$ ./gradlew gem  # -t to watch change of files and rebuild continuously