ecintelligence/yonder repository overview

⁠Yonder

Cold VM migrations to OpenStack, Proxmox VE, KubeVirt, and Nutanix AHV.

Yonder performs cold migrations of VMs between virtualization platforms: two sources, four targets. Distributed freely as two public Docker images; the source code is not publicly available. Support, deployment, and custom development are commercial services from EC Intelligence.

Sources: VMware vSphere / ESXi · OpenStack (since 1.0.0, validated since 1.1.0)

Targets: OpenStack (Nova/Glance/Cinder/Neutron) · Proxmox VE · KubeVirt / OpenShift Virtualization · Nutanix AHV (Prism Central, v4 API)

The source VM is shut down and its disks acquired per platform — VMware: consistent snapshot, disks over NFC HTTPS; OpenStack: exported through Glance (volume-backed via temporary snapshot so the source volume is never modified; ephemeral root via instance snapshot). Disks are converted to qcow2, adapted with virt-v2v / virt-customize (initramfs, guest networking), then uploaded/imported on the target.

⁠Images

Two images, both required. The conversion image is built from the main image at the same version — keep tags aligned (:1.5.0 with :1.5.0).

Latest version: 1.5.0

⁠Architecture

7 services via Docker Compose: backend (Django+Gunicorn, REST/JWT), celery-worker (async), celery-worker-conversion (conversion + guest fixup, uses yonder-conversion), celery-beat (scheduler + periodic temp-dir cleanup), nginx (SPA + /api/ proxy), db (PostgreSQL 14), redis (broker/result).

⁠Prerequisites

• Docker + Docker Compose plugin; outbound access to Docker Hub
• A free HTTP port (default 8077)
• Storage for the migration working dir (see Storage) — by default a Docker volume under /var/lib/docker
• Proxmox targets: the target node must reach the Yonder host over HTTP (the node pulls the disk; see Targets → Proxmox VE)
• Nutanix AHV targets: a reachable Prism Central (v4 API, GA on pc.2024.3 / AOS 7.0+), reachable from PC over HTTP for the same reason
• Optional: /dev/kvm on the host to accelerate the guest fixup (see KVM acceleration)

⁠Quick start

⁠1. Directory

mkdir -p /opt/yonder && cd /opt/yonder

⁠2. docker-compose.yml

services:
  nginx:
    image: nginx:alpine
    ports:
      - "${NGINX_HTTP_PORT}:80"
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
      - static_data:/data/static:ro
      - media_data:/data/media:ro
      - frontend_data:/data/frontend:ro
    extra_hosts:
      - "host.docker.internal:host-gateway"
    depends_on:
      - backend
    restart: unless-stopped

  backend:
    image: ecintelligence/yonder:${YONDER_VERSION:-latest}
    network_mode: host
    cap_add:
      - NET_RAW
    env_file: .env
    volumes:
      - static_data:/data/static
      - media_data:/data/media
      - frontend_data:/data/frontend
      - /proc:/host/proc:ro
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:${GUNICORN_PORT}/api/v1/health/"]
      interval: 30s
      timeout: 10s
      retries: 3
    restart: unless-stopped

  celery-worker:
    image: ecintelligence/yonder:${YONDER_VERSION:-latest}
    network_mode: host
    cap_add:
      - NET_RAW
    command: ["celery", "-A", "yonder_platform", "worker", "-Q", "default", "--loglevel=info", "--concurrency=${CELERY_CONCURRENCY}"]
    env_file: .env
    volumes:
      - /proc:/host/proc:ro
    depends_on:
      - backend
    restart: unless-stopped

  celery-worker-conversion:
    # To accelerate the guest fixup with KVM, do NOT add "devices:" here —
    # use the separate override file (see KVM acceleration).
    image: ecintelligence/yonder-conversion:${YONDER_VERSION:-latest}
    network_mode: host
    command: ["celery", "-A", "yonder_platform", "worker", "-Q", "conversion", "--loglevel=info", "--concurrency=${CELERY_CONVERSION_CONCURRENCY:-1}"]
    env_file: .env
    environment:
      MIGRATION_TEMP_ROOT: /mnt/migration_temp
    volumes:
      - /proc:/host/proc:ro
      # Default: Docker named volume (under /var/lib/docker).
      - migration_temp:/mnt/migration_temp
      # To relocate, comment the line above and bind-mount a host path (see Storage):
      # - /your/working/path:/mnt/migration_temp
    depends_on:
      - backend
    restart: unless-stopped

  celery-beat:
    image: ecintelligence/yonder:${YONDER_VERSION:-latest}
    network_mode: host
    command: ["celery", "-A", "yonder_platform", "beat", "--loglevel=info", "--scheduler", "django_celery_beat.schedulers:DatabaseScheduler"]
    env_file: .env
    volumes:
      - /proc:/host/proc:ro
    depends_on:
      - backend
    restart: unless-stopped

  db:
    image: postgres:14-alpine
    environment:
      POSTGRES_DB: ${DB_NAME}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - db_data:/var/lib/postgresql/data
    ports:
      - "127.0.0.1:5434:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DB_USER} -d ${DB_NAME}"]
      interval: 10s
      retries: 5
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    volumes:
      - redis_data:/data
    ports:
      - "127.0.0.1:6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      retries: 5
    restart: unless-stopped

volumes:
  db_data:
  redis_data:
  static_data:
  media_data:
  frontend_data:
  migration_temp:

⁠3. nginx.conf

upstream backend { server host.docker.internal:8011; }

server {
    listen 80;
    server_name _;
    client_max_body_size 100M;

    location /api/ {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }
    location /admin/ {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }
    location /static/ { alias /data/static/; expires 30d; access_log off; }
    location /media/  { alias /data/media/;  expires 30d; }
    location / {
        root /data/frontend;
        index index.html;
        try_files $uri $uri/ /index.html;
    }
    location ~* ^/assets/.+\.(js|css|woff2?|svg|png|jpg|jpeg|gif|ico)$ {
        root /data/frontend;
        expires 1y;
        add_header Cache-Control "public, immutable";
        access_log off;
    }
    location = /index.html {
        root /data/frontend;
        add_header Cache-Control "no-cache, no-store, must-revalidate";
        expires 0;
    }
    location = /favicon.svg { root /data/frontend; expires 7d; access_log off; }
}

⁠4. .env

Replace every CHANGE_ME before exposing the instance.

YONDER_VERSION=1.5.0

# Django core
DJANGO_SETTINGS_MODULE=yonder_platform.settings.production
SECRET_KEY=CHANGE_ME_LONG_RANDOM_50_CHARS
DEBUG=False
ALLOWED_HOSTS=<HOST_IP_OR_DOMAIN>
SITE_URL=http://<HOST_IP_OR_DOMAIN>:8077

# Database
DB_HOST=localhost
DB_PORT=5434
DB_NAME=yonder
DB_USER=yonder
DB_PASSWORD=CHANGE_ME_STRONG_PASSWORD

# Redis / Celery
CELERY_BROKER_URL=redis://localhost:6379/0
CELERY_RESULT_BACKEND=redis://localhost:6379/0
CELERY_DEFAULT_QUEUE=default
CELERY_CONCURRENCY=4
CELERY_CONVERSION_CONCURRENCY=1

# Gunicorn / Nginx
GUNICORN_PORT=8011
GUNICORN_WORKERS=4
NGINX_HTTP_PORT=8077

# JWT
JWT_ACCESS_LIFETIME_MINUTES=15
JWT_REFRESH_LIFETIME_DAYS=7

# Superuser bootstrap (first boot only; blank to disable)
[email protected]
DJANGO_SUPERUSER_PASSWORD=CHANGE_ME_ADMIN_PASSWORD

# Fernet key (credential encryption). Never change after creating credentials.
YONDER_FERNET_KEY=CHANGE_ME_FERNET_KEY

# Localisation
DJANGO_LANGUAGE_CODE=fr-fr
DJANGO_TIME_ZONE=UTC

# System introspection
SYSTEM_EXCLUDED_INTERFACES=lo,docker,br-,veth
SYSTEM_MOUNT_TYPES_INCLUDED=nfs,nfs4,cifs,ceph,fuse.glusterfs,ext4,xfs,btrfs,zfs

# Guest fixup (opt-in; default OFF)
MIGRATION_V2V_FIXUP=true
MIGRATION_L3_FIXUP=true
MIGRATION_OS_GUEST_NETPLAN=true

# KVM acceleration of the guest fixup appliance (1.5.0).
# off (default) | auto | on -- see the KVM acceleration section before enabling.
# MIGRATION_KVM_ACCEL=off

# Disk preflight margin (default 2.0; 0 disables)
# MIGRATION_PREFLIGHT_DISK_MARGIN=2.0

# Operation timeouts (optional; seconds). Defaults preserve prior behaviour.
# OPENSTACK_VOLUME_AVAILABLE_TIMEOUT_S=900 / OPENSTACK_IMAGE_ACTIVE_TIMEOUT_S=900
# OPENSTACK_SERVER_ACTIVE_TIMEOUT_S=600 / OPENSTACK_VOLUME_DELETE_TIMEOUT_S=300
# PROXMOX_VM_RUNNING_TIMEOUT_S=600 / PROXMOX_TASK_POLL_TIMEOUT_S=1800
# NUTANIX_IMAGE_IMPORT_TIMEOUT_S=1800 / NUTANIX_TASK_POLL_TIMEOUT_S=1800
# NUTANIX_VM_POWER_ON_TIMEOUT_S=600

Generate secrets:

python3 -c "import secrets; print(secrets.token_urlsafe(50))"                         # SECRET_KEY
python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"  # YONDER_FERNET_KEY

⁠5–6. Pull & start

docker compose pull
docker compose up -d --force-recreate

First boot waits for the DB, applies migrations, creates the superuser, and copies the frontend into frontend_data (slower than later boots). --force-recreate also reloads .env.

⁠7. Verify

docker compose ps        # backend must show (healthy)
curl -s http://<HOST_IP_OR_DOMAIN>:8077/api/v1/version/ && echo
curl -s http://<HOST_IP_OR_DOMAIN>:8077/api/v1/health/ && echo
curl -s http://<HOST_IP_OR_DOMAIN>:8077/api/v1/readyz/ && echo

Expected:

{"version":"1.5.0"}
{"status":"ok","version":"1.5.0","components":{"database":"ok","redis":"ok","celery_workers":"ok","fernet":"ok"}}
{"status":"ready","components":{"database":"ok","redis":"ok","celery_workers":"ok","fernet":"ok"}}

UI at http://<HOST_IP_OR_DOMAIN>:8077. Sign in with the bootstrap superuser (login by email).

⁠Sources

A source is where VMs migrate from. You select the type; the form asks only the needed fields and filters the credential picker. Discovered inventory is browsable with server-side search (name / guest OS), power-state filter, and column sorting.

⁠Targets

A target is where VMs migrate to. Select the type; the form filters fields and credential picker accordingly.

⁠Proxmox VE

No SSH since 1.3.0 — the whole path is the Proxmox REST API. The qcow2 is staged with download-url on an import-capable storage, then materialised with import-from; both are async node tasks Yonder polls.

• Node placement: a default node per target, overridable per migration.
• Network requirement: the target node pulls the disk over HTTP from the Yonder host. An intercepting proxy on that path breaks the import — exclude the Yonder host from any datacenter proxy.
• Hardware selectors (1.4.0), per migration: SCSI controller (scsihw) and NIC model.

⁠Nutanix AHV

Driven through Prism Central with the official Nutanix v4 SDKs. Requires pc.2024.3 / AOS 7.0+ (the v4 vmm API is Prism-Central-only; standalone Prism Element is not supported).

• Host = Prism Central address, port 9440. Cluster = the AHV compute cluster name registered in PC.
• In the mapping, storage maps to a Nutanix storage container (by name), network to an AHV subnet (by name).

Flow: the qcow2 is registered as a PC image via URL source (PC pulls it from the conversion worker over an ephemeral, token-scoped HTTP endpoint), the VM is created in one call (disks cloned from the images, NICs on the chosen subnet, UEFI/Legacy carried over, source MAC propagated), then powered on.

⁠Target network options (OpenStack)

Per-mapping: static fixed IP per NIC (realised on the Neutron port at creation; subnet resolved from the network CIDR) and security group per NIC (existing group, by name). Source MAC is propagated to the port. Intermediate Glance images used to build the Cinder volumes are auto-cleaned after the server reaches ACTIVE (fixed in 1.4.1).

⁠Storage (migration working directory)

Yonder converts source disks into /mnt/migration_temp (conversion worker) before upload. By default a Docker named volume under /var/lib/docker — i.e. on the root filesystem, not /opt/yonder. Too small → migration fails with [Errno 28] No space left on device.

To relocate onto sized/network storage:

1. Mount your storage on the host (Yonder does not mount it for you).
2. In celery-worker-conversion, replace the named volume with a bind mount — keep the right side /mnt/migration_temp:

       volumes:
         - /proc:/host/proc:ro
         # - migration_temp:/mnt/migration_temp     <- comment out
         - /your/mount/point:/mnt/migration_temp     <- your host path
   

   Copy
3. The worker writes as UID 1001 — the dir must be writable by it (on NFS check root_squash/mapping).
4. docker compose up -d --force-recreate, wait for healthy.

Conversion (qemu-img, virt-v2v) is I/O-intensive; test one full migration on NFS before production. A disk preflight refuses a migration up front if the working dir lacks space; finished working dirs are purged automatically.

⁠KVM acceleration (optional, since 1.5.0)

The guest fixup boots a libguestfs appliance (virt-v2v-in-place, virt-customize). Without /dev/kvm in the conversion worker it runs under software emulation (TCG). Measured on a 2.27 GB disk: virt-v2v ~300 s and L3 fixup 60–100 s, against ~10 s for the qemu-img conversion of the same disk — the fixup, not the data copy, dominates the migration window.

Why a probe and not just a device check: KVM being permitted does not mean KVM works. We measured a host where the device was mapped and accessible, libguestfs reported qemu KVM: enabled, and QEMU then aborted while starting the appliance (the kernel refused a write to MSR 0x345). The same test succeeded on a host with a newer kernel. Leaving TCG on such a host does not make the fixup slower, it makes it fail — so any doubt resolves to TCG.

⁠Checks before enabling

# 1. Device present? Note the gid -- it is host-specific.
stat -c 'owner=%U group=%G gid=%g mode=%a' /dev/kvm && getent group kvm

# 2. Does an appliance actually boot under KVM here? Replace <GID>.
docker run --rm --user 1001 --group-add <GID> --device /dev/kvm:/dev/kvm \
  --entrypoint "" ecintelligence/yonder-conversion:latest \
  sh -c 'LIBGUESTFS_BACKEND_SETTINGS=force_kvm libguestfs-test-tool >/tmp/t.log 2>&1; \
         echo "rc=$?"; grep -iE "qemu KVM:|failed to set MSR|Assertion" /tmp/t.log'

Expected for check 2: rc=0 and libguestfs: qemu KVM: enabled. A non-zero rc means KVM is not usable for the appliance on this host — keep off.

⁠Enabling

Create docker-compose.kvm.yml next to docker-compose.yml:

services:
  celery-worker-conversion:
    devices:
      - "/dev/kvm:/dev/kvm"
    group_add:
      - "${YONDER_KVM_GID:?set YONDER_KVM_GID in .env to the host kvm group id (getent group kvm)}"

Add to .env (replace <GID> with the value from check 1):

COMPOSE_FILE=docker-compose.yml:docker-compose.kvm.yml
YONDER_KVM_GID=<GID>
MIGRATION_KVM_ACCEL=auto

Then docker compose up -d --force-recreate celery-worker-conversion. The guest fixup phase now emits a job milestone stating the mode and the effective regime, with the reason when acceleration was not obtained.

To disable: remove those three lines and recreate. Setting MIGRATION_KVM_ACCEL=off also forces software emulation even with the device mapped.

⁠Notes

• group_add is mandatory. /dev/kvm is root:kvm 0660 and the worker runs non-root: mapping the device alone yields EACCES, the platform falls back to TCG, and you get no acceleration while believing the fix is in place.
• devices: must stay out of the main compose file. A hard entry makes the container refuse to start on any host without /dev/kvm.
• The group may display under another name inside the container. The kernel only checks the numeric gid, so /dev/kvm showing as owned by e.g. clock is expected and harmless.
• Adding COMPOSE_FILE does not change the compose project name — existing volumes are preserved.
• Recreating the conversion worker also restarts backend; expect a short 502 until it reports healthy.

⁠Configuration reference

Timeout variables are optional; defaults preserve normal behaviour — override only for slow storage/clouds.

Health endpoints (auth-free): /api/v1/version/ (running version) · /api/v1/health/ (liveness, always 200; reports database/redis/celery_workers/fernet) · /api/v1/readyz/ (readiness; 200 ready, 503 otherwise).

Ports: 8077 HTTP/UI (nginx) · 8011 Gunicorn (host net) · 5434 PostgreSQL (127.0.0.1) · 6379 Redis (127.0.0.1).

⁠Upgrading

Pull-only — no local build:

cd /opt/yonder && docker compose pull && docker compose up -d --force-recreate

Wait for the backend healthy. If you pin a version, bump YONDER_VERSION first. After editing .env, always up -d --force-recreate (a plain restart does not re-read it).

⁠Tags

⁠Links & support

Main image: https://hub.docker.com/r/ecintelligence/yonder⁠ · Conversion image: https://hub.docker.com/r/ecintelligence/yonder-conversion⁠

Support: [email protected]⁠ · https://ecintelligence.ma⁠ · Maintained by EC Intelligence.