The tweakstreet data engineering tool
1.8K
The image contains the tweakstreet execution engine. It makes all Tweakstreet CLI commands available on PATH.
The image has an /app folder which is a convenient place to mount your project files.
The following runs /my/project/main.dfl using engine.sh, outputs the logs and removes the resulting container when done.
$ docker run --init --rm \
-v /my/project:/app \
twineworks/tweakstreet \
engine.sh /app/main.dfl
2020-06-19 18:28:04 Z | INFO | main | loading flow /app/main.dfl
2020-06-19 18:28:04 Z | INFO | main.dfl | initializing flow
...
2020-06-19 18:28:06 Z | INFO | write records | FINISHED total: {"in": 123, "out": 123, "error": 0}
2020-06-19 18:28:06 Z | INFO | main.dfl | flow finished successfully
If you wish to run a long-running job in the background use the -d and --name docker switches:
$ docker run --init -d \
--name long_job \
-v /my/project:/app \
twineworks/tweakstreet \
engine.sh /app/main.dfl
Docker will run the container in the background. You can output the logs of your container using docker logs or follow them with docker logs -f :
$ docker logs long_job
2020-06-19 18:28:04 Z | INFO | main | loading flow /app/main.dfl
2020-06-19 18:28:04 Z | INFO | main.dfl | initializing flow
...
2020-06-19 18:28:06 Z | INFO | write records | FINISHED total: {"in": 123, "out": 123, "error": 0}
2020-06-19 18:28:06 Z | INFO | main.dfl | flow finished successfully
Don't forget to remove the container once it is no longer needed to free up disk space.
$ docker rm long_job
The image contains a /home/tweakstreet folder. Mount a folder or volume at /home/tweakstreet/.tweakstreet if you need to provide drivers or plugins.
$ docker run --init --rm \
-v /my/.tweakstreet:/home/tweakstreet/.tweakstreet:ro \
-v /my/project:/app \
twineworks/tweakstreet \
engine.sh /app/main.dfl
Commands run as user tweakstreet in main group tweakstreet with uid:gid set to 1001:1001.
The tweakstreet binaries are in /opt/tweakstreet/bin
Content type
Image
Digest
sha256:c5b3f4fe4…
Size
198 MB
Last updated
over 2 years ago
docker pull twineworks/tweakstreet