dgunchev/devpi repository overview

⁠docker-devpi

This repository contains a Dockerfile for devpi pypi server⁠.

You can use this container to speed up the pip install parts of your docker builds. This is done by adding an optional cache of your requirement python packages and speed up docker. The outcome is faster development without breaking builds.

⁠Getting started

⁠Installation

docker pull dgunchev/devpi

⁠Quickstart

Start using

docker run -d --name devpi \
    --publish 3141:3141 \
    --volume /srv/docker/devpi:/data \
    --env=DEVPI_PASSWORD=changemetoyourlongsecret \
    --restart always \
    dgunchev/devpi

Alternatively, you can use the sample docker-compose.yml⁠ file to start the container using Docker Compose⁠

Please set DEVPI_PASSWORD to a secret otherwise an attacker can execute arbitrary code.

⁠Client side usage

To use this devpi cache to speed up your dockerfile builds, add the code below in your dockerfiles. This will add the devpi container an optional cache for pip. The docker containers will try using port 3141 on the docker host first and fall back on the normal pypi servers without breaking the build.

For Debian/Ubuntu/deb based images

# Install iproute2
RUN apt-get update \
 && apt-get install -y iproute2 \
 && rm -rf /var/lib/apt/lists/*

 # Use an optional pip cache to speed development
RUN export HOST_IP=$(ip route| awk '/^default/ {print $3}') \
 && mkdir -p ~/.pip \
 && echo [global] >> ~/.pip/pip.conf \
 && echo extra-index-url = http://$HOST_IP:3141/app/dev/+simple >> ~/.pip/pip.conf \
 && echo [install] >> ~/.pip/pip.conf \
 && echo trusted-host = $HOST_IP >> ~/.pip/pip.conf \
 && cat ~/.pip/pip.conf

For new Fedora/Redhat/rpm+dnf based images

# Install iproute
RUN dnf install iproute \
 && dnf clean all

 # Use an optional pip cache to speed development
RUN export HOST_IP=$(ip route| awk '/^default/ {print $3}') \
 && mkdir -p ~/.pip \
 && echo [global] >> ~/.pip/pip.conf \
 && echo extra-index-url = http://$HOST_IP:3141/app/dev/+simple >> ~/.pip/pip.conf \
 && echo [install] >> ~/.pip/pip.conf \
 && echo trusted-host = $HOST_IP >> ~/.pip/pip.conf \
 && cat ~/.pip/pip.conf

For old Fedora/Redhat/rpm+yum based images

# Install iproute
RUN yum install iproute \
 && yum clean all

 # Use an optional pip cache to speed development
RUN export HOST_IP=$(ip route| awk '/^default/ {print $3}') \
 && mkdir -p ~/.pip \
 && echo [global] >> ~/.pip/pip.conf \
 && echo extra-index-url = http://$HOST_IP:3141/app/dev/+simple >> ~/.pip/pip.conf \
 && echo [install] >> ~/.pip/pip.conf \
 && echo trusted-host = $HOST_IP >> ~/.pip/pip.conf \
 && cat ~/.pip/pip.conf

The default IP can be found using python too:

python2 -c 'import socket; sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM); \
    sock.connect(("8.8.8.8", 1)); print(sock.getsockname()[0])'

⁠Uploading python packages files

You need to upload your python requirement to get any benefit from the devpi container. You can upload them using the bash code below a similar build environment.

pip wheel --download=packages --wheel-dir=wheelhouse -r requirements.txt
pip install "devpi-client>=2.3.0" \
&& export HOST_IP=$(ip route| awk '/^default/ {print $3}') \
&& if devpi use http://$HOST_IP:3141>/dev/null; then \
       devpi use http://$HOST_IP:3141/root/public --set-cfg \
    && devpi login root --password=$DEVPI_PASSWORD  \
    && devpi upload --from-dir --formats=* ./wheelhouse ./packages; \
else \
    echo "No started devpi container found at http://$HOST_IP:3141"; \
fi

⁠Persistence

For devpi to preserve its state across container shutdown and startup you should mount a volume at /data. The quickstart command already includes this.

⁠Security

Devpi creates a user named root by default, its password should be set with DEVPI_PASSWORD environment variable. Please set it, otherwise attackers can execute arbitrary code in your application by uploading modified packages.

For additional security the argument --restrict-modify root has been added so only the root may create users and indexes.

⁠History

This is a fork the Centre for Comparative Genomics, Murdoch University⁠'s docker-devpi⁠ repository, which was last updated 2018-05-24 (2.5 years ago). I needed something fresher so I forked it.

I managed to get the image size down to 29% of the original size. Initially it used the python:3.9.1 image and resulted in 994 MB image. Switching to fedora:latest resulted in 493 MB. Adding dnf clean all the image is now down to 282 MB (or ~ 90 MB compressed).

Building with alpine linux got the image down to 135 MB, 7.3 times smaller. The compressed image is 54.45 MB. This was a good excercise - building in one container with gcc, copying the compiled files and install them in another container without gcc... cool. Also removed 1.9 MB of APKINDEX gz caches.