vicoslab/ccc repository overview

⁠Conda Compute Containers

Compute containers that use Conda in users directory (that should be mounted as a persistent volume or shared folder) for further package installation, while the image itself is not writeable by the user. Images also contain basic labels that are used to configure FRP port proxying.

Three image TYPEs exist currently:

• base (Dockerfile⁠)
• vscode (Dockerfile⁠)
• jupyter (Dockerfile⁠) (>=v1.09 supported only on Ubuntu 22.04)
• x2go (Dockerfile⁠)
• xpra (Dockerfile⁠) (>=v1.11 supported only on Ubuntu 22.04 or newer)
• (deprecated since v1.09) jetbrains-projector (Dockerfile⁠)

Each image TPYE one is build for different versions of nvidia/cuda:XYZ-cudnn8-devel-ubuntu22.04. Image names for base type are below (replace base for other types):

• vicoslab/ccc:base-v1.12-ubuntu22.04-cuda12.3.2
• vicoslab/ccc:base-v1.12-ubuntu22.04-cuda12.4.1
• vicoslab/ccc:base-v1.12-ubuntu22.04-cuda12.5.1
• vicoslab/ccc:base-v1.12-ubuntu22.04-cuda12.6.1
• vicoslab/ccc:base-v1.12-ubuntu22.04-cuda13.0.2

Latest CCC build version is v1.12. See change log below.

The following legacy versions are also supported:

• up to version v1.11 (ubuntu 20.04):
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.0.3
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.1.1
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.2.2
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.3.1
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.5.2
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.7.1
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda11.8.0
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda12.0.0
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda12.1.0
  • vicoslab/ccc:base-v1.11-ubuntu20.04-cuda12.2.2
• up to version v1.07 (ubuntu 18.04):
  • vicoslab/ccc:base-v1.07-ubuntu18.04
  • vicoslab/ccc:base-v1.07-ubuntu18.04-cuda10.0
  • vicoslab/ccc:base-v1.07-ubuntu18.04-cuda10.1
  • vicoslab/ccc:base-v1.07-ubuntu18.04-cuda10.2
  • vicoslab/ccc:base-v1.07-ubuntu18.04-cuda11.0.3
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.0.3
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.1.1
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.2.2
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.3.0
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.5.0
  • vicoslab/ccc:base-v1.07-ubuntu20.04-cuda11.7.0

⁠Base image

The image uses runit for bootstrap, it runs rsyslogd, crond and sshd. It also runs several startup scripts that:

• configures user with ssh pubkey if not present,
• configures SSH access with pubkey only and creates host keys at /home/.sshd/keys if not present
• installs miniconda to ~/conda,
• installs optional APT packages (and sources),
• installs ccc-tools to system-wide path (/usr/bin/ccc)
• optionally reverts ENVs to last working version if startup failed 3-times, and
• provides welcome message (MOTD) (info on installed packages, etc)

Required environmental variables:

• USER_NAME: the name of the user that will use the container, defaults to user.
• USER_ID: the UID of the user, defaults to 1000.
• USER_PUBKEY: the public part of the key for SSH access.

Optional environmental variables:

• INSTALL_PACKAGES: apt-get packages that will be installed at startup-time.
• INSTALL_REPOSITORY_KEYS: comma separated list of URLs to GPG keys that will be installed at startup-time using wget -qO - "<KEY>" | apt-key add -.
• INSTALL_REPOSITORY_SOURCES: comma separated list of sources that will be installed at startup-time using add-apt-repository -y "<SOURCE>".
• RUNIT_STATUS_FILE: path to file where container will report its boot status (e.g., running, booting [NUM], failed boot)
• RUNIT_WORKING_ENV_FILE: path to file where last working ENVs are stored (for recovery if invalid ENV values supplied)

⁠VS Code

Extends base image and installs a VS Code IDE. It starts code serve-web service on default port 9999 (or one specified in VSCODE_PORT env var). The instance is protected using a token that is saved in file ~/vscode.token. Token is also displayed at the end of welcome message in SSH.

Access VS Code IDE in browser at https://your-url.com/?tkn=[TOKEN⁠], where [TOKEN] is value saved in ~/vscode.token and displayed at the end of welcome message in SSH.

⁠Jupyter

Extends base image, contains a Jupyter instance that is run at startup. The instance is protected using a token that is saved in file ~/jupyter.token. Token is also displayed at the end of welcome message in SSH.

⁠X11 support with XFCE4 and X2Go

Extends base image with XFCE4 and X2Go server, which is run when client connects through the SSH. Image also installs mate-terminal krusader ristretto gvfs gvfs-backends thunar-archive-plugin kio-extras ktexteditor-katepart packages and default themes with xubuntu-artwork xubuntu-icon-theme breeze-icon-theme packages.

Additional Environmental variables:

• APPS: installs symlinks to /usr/local/bin (paths separated by :)

⁠X11 support with XFCE4 and XPRA HTML5 server

Extends base image with XFCE4 and X2Go server, which is run when client connects through the SSH. Image also installs mate-terminal krusader ristretto gvfs gvfs-backends thunar-archive-plugin kio-extras ktexteditor-katepart packages and default themes with xubuntu-artwork xubuntu-icon-theme breeze-icon-theme packages.

Provides XPRA HTML5 proxy server that enables simple entry point for running and displaying any X11 application. HTML5 provides a simple access through browser since there are no dependency on client side other than HTML5 browser with web-socket support.

Additional Environmental variables:

• APPS: installs symlinks to /usr/local/bin (paths separated by :)
• XPRA_PROXY_HTML5_PORT: optional local port where xpra proxy is bound (defaults to 8888)

⁠JetBrains Projector support with XPRA HTML5 server

Extends XPRA HTML5 server image with JetBrains Projector⁠ server, which provides a remote access to IDE (pycharm, clion, etc.) running on a server accessible directly through the browser.

The instance is protected using a token that is displayed in SSH welcome message and in ~/.projector/configs/<config-name>/config.ini. NOTE: Updating token in config.in requires update of the config by running conda run -n jetbrains-projector projector update rebuild and rebooting the container.

Additional Environmental variables:

• PROJECTOR_IDE: optional ide name installed at first use (defaults to "PyCharm Professional Edition 2021.1.3")
• PROJECTOR_PORT: optional local port where JetBrains Projector is bound (defaults to 9999)
• PROJECTOR_CONFIG: optional name of the projector configuration (defaults to 'default')

⁠CHANGELOG:

Changes for v1.12:

• implemented logging to /home/$USER/.containers/
• fixed XPRA version for Ubuntu22.04
• depricated Ubuntu20.04 build
• added CUDA 13

Changes for v1.11:

• modified ccc-tools install to atomic operations (wait for install to finish from other containers)
• added CCC_TOOLS_ENABLE env to disable ccc-tools installation
• updated xpra to latest LTS version 5.1.1-r0-1

Changes for v1.10:

• bug fixed (ccc-tools issues when restarting container, apt install issues in post-script cleanup)

Changes for v1.09:

• added ccc-tools (https://github.com/vicoslab/ccc-tools⁠) to base image
• added VS Code Web-Server IDE version

Changes for v1.08:

• fixed bug when install latest version of miniconda

Changes for v1.07:

• fixed bug when install latest version of miniconda
• added mamba by default during miniconda install

Changes for v1.06:

• fixed bug when existing public key were overridden
• added CUDA_DEVICE_ORDER=PCI_BUS_ID to ensure order of devices will match nvidia-smi
• added CUDA_VISIBLE_DEVICES=-1 to disable visibility of devices by default and forcing users to properly set them

Changes for v1.05:

• upgraded cuda images to latest version with new GPG key
• enabled ignoring of errors during apt key update in case GPG keys of any existing packages changes
• bug fixes for jupyter and x2go/xpra for compatibility with latest packages

Changes for v1.04:

• added installation of latest ca-certificate upon start
• added default user/pass for xpra html5 server

Changes for v1.03:

• added ~/.services folder as symlink to allow simultainous running of user services on multiple machines

Changes for v1.02:

• added new jetbrains projector image that starts projector server at default port 9999
• updated X11 for xpra to start HTML5 server (xpra-proxy) at default port 8888
• updated base image to include libs related to mesa (libxext6 libsm6 libxrender1 libgl1-mesa-glx libglu1-mesa libxt6 libxtst6 libxi6)
• updated base image to include tools for ssh (nano htop tmux curl)
• added option to assign groups to the user through USER_GROUPS environment variable
• added jupyter token to MOTD at login
• minor bugfixes and code cleanup

Changes for v1.01:

• added loading of default container environment variables from /etc/envvars in /etc/profile.d/00_container_envs.sh
• added option to specify app name in APPS env for x11 containers
• updated welcome message with docker image, container name and any additional user supplied msg (CONTAINER_NAME, CONTAINER_IMAGE and CONTAINER_WELCOME_MSG envs must be provided now)
• removed saving HOME HOSTNAME TERM PWD USER USERNAME _ in/etc/envvars
• removed newer flag in dbus run to make it compatible with ubuntu 16.04