Data Science Docker Image
This image is created from the official Ubuntu 14.04 Docker image and contains popular Python packages for data science.
This repository defines the "civisanalytics/datascience-python"
Docker image. This Docker image provides an environment with data science tools
from the Python ecosystem. This image is the execution environment for Python
jobs in the Civis data science platform,
and it includes the Civis Python API client.
Either build the Docker image locally
docker build -t datascience-python .
or download the image from DockerHub
docker pull civisanalytics/datascience-python:latest
latest tag (Docker's default if you don't specify a tag)
will give you the most recently-built version of the datascience-python
image. You can replace the tag
latest with a version number such as
to retrieve a reproducible environment.
Inside the datascience-python Docker image, Python packages are installed in the
environment. For a full list of included Python libraries, see the
To start a Docker container from the datascience-python image and
interact with it from a bash prompt, use
docker run -i -t civisanalytics/datascience-python:latest /bin/bash
You can run a Python command with
docker run civisanalytics/datascience-python:latest python -c "import pandas; print(pandas.__version__)"
The image contains environment variables which allow you to find
the current version. There are four environment variables defined:
VERSION VERSION_MAJOR VERSION_MINOR VERSION_MICRO
VERSION contains the full version string, e.g. "1.0.3". VERSION_MAJOR,
VERSION_MINOR, and VERSION_MICRO each contain a single integer.
Joblib Temporary Files
joblib library enhances multiprocessing
capabilities for scientific Python computing. In particular, the
joblib for parallelization. This Docker image sets
default location for staging temporary files to the /tmp directory.
The normal default is /shm. /shm is a RAM disk which defaults to a 64 MB size
in Docker containers, too small for typical scientific computing.
Creating Equivalent Local Environments
environment.yml file in this repo can be used to create a python environment that is
equivalent to the one in the container. This environment will be named
The environment installs in Ubuntu Linux (this is the OS of the Dockerfile).
It will install in OS X, but the
xgboost install requires either
gcc v5 or the
clang-omp compiler, neither of which are natively provided in OS X.
If you wish to set up this environment in OS X, you may either
environment.ymlfile before using it to create the environment
- Use Homebrew to install
gcc-5. You can do that via
brew install gcc@5 --without-multilib. Be warned that this installation will take
a long time.
See CONTRIBUTING for information about contributing to this project.
If you make any changes, be sure to build a container to verify that it successfully completes:
docker build -t datascience-python:test .
and describe any changes in the change log.
This repo has autobuild enabled. Any PR that is merged to master will
be built as the
latest tag on Dockerhub.
Once you are ready to create a new version, go to the "releases" tab of the repository and click
"Draft a new release". Github will prompt you to create a new tag, release title, and release
description. The tag should use semantic versioning in the form "vX.X.X"; "major.minor.micro".
The title of the release should be the same as the tag. Include a change log in the release description.
Once the release is tagged, DockerHub will automatically build three identical containers, with labels
"major", "major.minor", and "major.minor.micro".
See LICENSE.md for details.