Sign inSign up

jonaspammerwork/python

By jonaspammerwork

•Updated 8 months ago

Image
0

2.4K

jonaspammerwork/python repository overview

⁠python:3.12 (ubuntu) CI/CD Development Image

This image is maintained solely for use by myself in CI/CD pipelines. It's purpose is to pre-package environment variables and install tools i need to reduce runtime and Lines of Code.

Because they are inherently large in size, its recommended to use a dependency proxy:

    - echo "$CI_DEPENDENCY_PROXY_PASSWORD" | docker login $CI_DEPENDENCY_PROXY_SERVER -u $CI_DEPENDENCY_PROXY_USER --password-stdin || echo "Failed to login to dependency proxy, continuing..."

⁠Common Features

  • Uses Multi-Stage Build to copy binaries and files from the following official images:

    • Docker-CLI
    • Selenium/Chrome binaries This is done as to make them figure out how to best install it inside docker.
  • Sets Environment variables for GitLab Runner's Docker-in-Docker (tcp://docker:2376)

  • Installs Common Unix utilities from apt, including: curl, wget, git, openssl, wait-for-it, dos2unix, bc, netcat-openbsd, ripgrep, gettext, and build tools (build-essential, gcc, pkg-config, binutils). Upgrades all system packages for increased security. Does not clear update cache.

  • Installs Taskfile (task) - Simpler Make alternative⁠ to /usr/bin

  • Installs swaks - Swiss Army Knife for SMTP⁠ to /usr/bin

  • Installs Ruby, gem, and bundler - because some Asciidoctor Tools require it.

  • Installs nvm - because my project's arent node based, but some things require node, so CI does not have to fall back to starting node container just for one off thing

⁠Image Specific Features

  • UV⁠: Fast Package and project manager for Python.

A single tool to replace pip, pip-tools, pipx, poetry, pyenv, twine, virtualenv, and more.

⁠gitlab-ci.yml Examples

Enable caching (taken from https://docs.astral.sh/uv/guides/integration/gitlab/⁠):

uv-install:
  image: jonaspammerwork/python
  cache:
    - key:
        files:
          - uv.lock
      paths:
        - $UV_CACHE_DIR
  script:
    # Your `uv` commands
    - uv cache prune --ci

Run a ruff format in parallel with a ruff check compatible with GitLab's codequality report. (taken from https://docs.astral.sh/ruff/integrations/#github-actions⁠)

.base_ruff:
  stage: build
  interruptible: true
  image:
    name: jonaspammerwork/python
  before_script:
    - cd $CI_PROJECT_DIR
    - ruff --version

ruff-check:
  extends: .base_ruff
  script:
    - ruff check --output-format=gitlab > code-quality-report.json
  artifacts:
    reports:
      codequality: $CI_PROJECT_DIR/code-quality-report.json

ruff-format:
  extends: .base_ruff
  script:
    - ruff format --diff

⁠Dockerfile

Click to expand
ARG BASE_IMAGE="python:3.12-bookworm"

FROM ghcr.io/astral-sh/ruff:latest AS ruff
FROM ghcr.io/astral-sh/uv:latest AS uv
FROM selenium/standalone-chrome AS chrome
FROM docker:27-cli AS docker-cli
FROM ${BASE_IMAGE:-ubuntu:latest}

## Common Features
ENV DEBIAN_FRONTEND=noninteractive
ENV LANG=C.UTF-8
ENV LC_ALL=C.UTF-8
# Enable e.g. 'source' and [[
SHELL ["/bin/bash", "-c"]
# Set up environment variables for Docker-in-Docker communication as per GitLab Runner
ENV DOCKER_HOST=tcp://docker:2376 \
    DOCKER_TLS_CERTDIR=/certs \
    DOCKER_TLS_VERIFY=1 \
    DOCKER_CERT_PATH=/certs/client
# Add docker binary and optional supporting files from the docker-cli image
COPY --from=docker-cli /usr/local/bin/docker /usr/local/bin/docker
COPY --from=docker-cli /usr/local/libexec/docker /usr/local/libexec/docker
# Add Chrome support
COPY --from=chrome /opt/selenium /opt/selenium
COPY --from=chrome /usr/bin/google-chrome /usr/bin/google-chrome
COPY --from=chrome /usr/bin/google-chrome /usr/bin/chromedriver
# choose fastest mirror (https://www.baeldung.com/linux/apt-terminal-choose-fastest-mirror)
RUN source /etc/os-release && \
    if [[ "$ID" == "ubuntu" ]]; then \
        sed -iv "s|deb [a-z]*://[^ ]* |deb mirror://mirrors.ubuntu.com/mirrors.txt |g" /etc/apt/sources.list; \
    fi;
# (Deliberately not removing apt cache)
RUN apt-get -qy update && apt-get -qy upgrade && apt-get -qy autoremove
RUN apt-get -qy install tzdata
# Install common Unix utilities
RUN apt-get -qy install curl tar zip unzip wget sed recode dos2unix rpl grep coreutils rsync git jq build-essential util-linux net-tools tree dnsutils iputils-ping
# Install useful CI tools
RUN apt-get -qy install openssl bc netcat-openbsd wait-for-it ripgrep gettext gcc pkg-config binutils
RUN apt-get -qqy install bat nano iproute2 || true
# Install "SMTP Swiss Army Knife"
RUN (curl -L https://jetmore.org/john/code/swaks/files/swaks-20240103.0/swaks -o /usr/bin/swaks && chmod 755 /usr/bin/swaks) || apt-get install -qy swaks
# Install Taskfile
RUN sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/bin
# Install Ruby, gem, and Bundler
RUN apt-get -qy install ruby && gem install bundler
# https://github.com/ruby/bigdecimal/issues/297 - fix bigdecimal install error
RUN apt-get -qy install libjemalloc-dev libgmp-dev libz-dev clang

# Install Node Version Manager
ENV NVM_DIR=/root/.nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/master/install.sh | bash


# Python Best Practices
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

# install python3-wheel, python3-dev and (for security purposes) upgrade python packages of system python
USER root
RUN export DEBIAN_FRONTEND=noninteractive && apt-get -qy update \
    && apt-get -qqy install --no-install-recommends python3-wheel python3-dev \
    && python3 -m pip install --no-cache-dir --upgrade pip \
    && python3 -m pip install --no-cache-dir --upgrade setuptools \
    && python3 -m pip install --no-cache-dir --upgrade wheel \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

# or: RUN curl -LsSf https://astral.sh/uv/install.sh | sh
COPY --from=uv /uv /uvx /usr/bin/
# or: RUN uv tool install ruff@latest
# or: RUN curl -LsSf https://astral.sh/ruff/install.sh | sh
COPY --from=ruff /ruff /usr/bin/

# Setup UV - https://docs.astral.sh/uv/guides/integration/gitlab/
ENV UV_CACHE_DIR=".uv-cache"
# GitLab CI creates a separate mountpoint for the build directory,
# so we need to copy instead of using hard links.
ENV UV_LINK_MODE="copy"

Tag summary

Content type

Image

Digest

sha256:b16dff7b7…

Size

837.4 MB

Last updated

8 months ago

docker pull jonaspammerwork/python