A ready-to-run Docker image with CLI tools for data engineering, ETL, and data science. Uses Ubuntu with zsh, and runs as user dataeng (password dataeng) in /home/dataeng.
Interactive shell (default):
docker pull lucascordova/dataeng
docker run -it --rm lucascordova/dataeng
With a directory mounted (e.g. project or data folder):
docker run -it --rm -v /path/on/host:/home/dataeng/work lucascordova/dataeng
One-off command (e.g. run jq on a file):
docker run --rm -v /path/on/host:/data lucascordova/dataeng jq '.' /data/file.json
Use your own .zshrc by mounting a directory and pointing the container at it:
docker run -it --rm \
-v /path/on/host:/mnt/data \
-e HOST_ZSHRC=/mnt/data/.zshrc \
lucascordova/dataeng
| Tool | Description |
|---|---|
| jq | JSON processor for the shell: query, filter, and transform JSON. |
| gron | Flatten JSON into discrete assignments so you can grep and reason line-by-line. |
| fx | Interactive JSON viewer and processor (Node). |
| xml2json-command | Convert XML to JSON from the command line (Node). |
| json2csv | Convert JSON (including nested) to CSV. |
| Tool | Description |
|---|---|
| xsv | Fast CSV toolkit: slice, split, select, stats, join, and more. |
| csvkit | Suite of CSV tools: in2csv, csvlook, csvstat, csvsql, csvjoin, etc. |
| csvquote | Wraps CSV so tools that don’t understand quoted fields (e.g. cut, sort) work correctly. |
| Tool | Description |
|---|---|
| xmlstarlet | Query, transform, and edit XML/HTML from the shell. |
| pup | Parse HTML and extract content using CSS selectors in the shell. |
| Tool | Description |
|---|---|
| GNU parallel | Run many jobs in parallel; great for looping over files or lines. |
| tldr | Short, example-focused “man” pages for common commands. |
| bat | cat with syntax highlighting and optional paging. |
| Tool | Description |
|---|---|
| Python 3 | With pip, venv, and pipx. Default python points to python3. |
| pydata-venv | Pre-created venv with numpy and pandas; auto-activated via /etc/profile.d/pydata-venv.sh. |
| pybase-venv | Venv with vowpalwabbit (fast online ML) and cssselect. |
| sample-stream | Reservoir sampling for streams (installed via pipx). |
| R | R with tidyverse, janitor, docopt, remotes, and devout/ANSI packages. |
| Node.js & npm | For JS tooling; fx and xml2json-command installed globally. |
| Rust | rustc and cargo for building Rust tools. |
| Tool | Description |
|---|---|
| vowpalwabbit | Fast online learning (in pybase-venv). |
| tapkee | Dimensionality reduction (e.g. PCA, t-SNE, LLE). |
| numpy / pandas | In pydata-venv for quick analysis. |
| Tool | Description |
|---|---|
| AWS CLI | Official AWS command-line interface (v2). |
| Tool | Description |
|---|---|
| dsutils | Collection of small data-science CLI utilities (from dsutils). |
| curl | Transfer data from or to a server (HTTP, HTTPS, FTP, etc.). |
| wget | Download files from the web; supports HTTP, HTTPS, and FTP, including recursive downloads. |
| git | Version control. |
| make | Build automation. |
| htop | Interactive process viewer. |
| tree | Directory listing as a tree. |
| nano | Simple editor. |
| pbcopy / pbpaste | Clipboard (backed by xclip) for copy/paste from the container. |
| cowsay | ASCII “cow” says your message. |
cmake, g++, build-essential, and common dev libraries (e.g. Boost, zlib, libcurl, libzip, Eigen) are installed so you can build C/C++ projects.
ubuntu:latestdataeng (password dataeng), with passwordless sudo/home/dataengFor source and build instructions, see the GitHub repo.
Content type
Image
Digest
sha256:d5971a8c7…
Size
804.4 MB
Last updated
7 months ago
docker pull lucascordova/dataeng