qualityfirstsoftware/qftest-interactive repository overview

⁠Image overview

The QF-Test container images come within two distinct flavours: minimal and interactive. The image tag corresponds to the version number of the included QF-Test installation.

⁠Web tests

For web tests, there are images including the latest stable chrome release, i.e. qftest-minimal-chrome and qftest-interactive-chrome.

⁠Quick start

⁠Interactive environment

For a quick start of the containerized QF-Test application, use a command like follows:

docker run -p 5901:5901 qualityfirstsoftware/qftest-interactive

This spawns a container from the qualityfirstsoftware/qftest-interactive image which will listen on port :5901 for incoming VNC connections. The default password is: qf-test.

If no vncviewer is available on your system, you can use the docker-compose⁠ file shown below, which will bring up (apart from the main QF-Test container) a second container hosting novnc⁠, a VNC viewer based on web technology. Within the specified compose stack, the viewer server is configured so that it can reach the interactive desktop and renders it as web application.

#docker-compose.yml

version: "3.9"
services:
  qf-test:
    image: "qualityfirstsoftware/qftest-interactive"
  vnc:
    image: "qualityfirstsoftware/novnc"
    ports:
      - "8081:8081"
    environment:
      - "REMOTE_HOST=qf-test"
      - "REMOTE_PORT=5901"

• Within the directory of the docker-compose.yml file, run it via: docker-compose up -d
• To access the container session, point your web browser to http://<DOCKER_HOST>:8081/vnc.html and establish the VNC connection by pressing the Connect button. If run locally, <DOCKER_HOST> translates to 127.0.0.1 (or localhost).
• In the end, stop the container services with docker-compose down.

⁠Step-by-step guide

We will use the internal CarConfigurator as an example how to setup a container-backed batch test of an test suite and finally write your own in interactive mode. Experienced container users can skip this section and consult the more formal specifications⁠.

⁠Batch calls

A normal, i.e. non-containerized batch test requires the command line option -batch. The corresponding QF-Test call would look like this: qftest -batch /path/to/your/suite. Within the container, we will use the internal demo suite, that is located at /qftest/qftest-container/demo/carconfigSwing/carconfigSwing_en.qft The container accepts exactly the same arguments that you would pass to a system installation. Thus, the corresponding docker command is:

docker run qualityfirstsoftware/qftest-interactive -batch /qftest/qftest-container/demo/carconfigSwing/carconfigSwing_en.qft

⁠The workdir: Persistence and data sharing

Once the container quits and gets removed, all runtime changes to the container file system (stored in the container layer on top of the actual image) are lost. Data persistence can be achieved by the means of volumes. A commonly used volume type is to bind mount a directory on the host system into the container. In the previous example, the runlog produced by running the CarConfigurator should be accessible even after the test has been run and the container stopped. Within the container, the directory /workdir serves as the workdir to the main QF-Test process. If not configured differently, runlogs will also appear in this directory. It is therefore a good candidate for being replaced by a volume (--volume|-v option).

# create a directory on the host site (if not existent)
mkdir -p container_workdir

# run the test suite with /workdir being a bind mount
docker run -v $(pwd)/container_workdir:/workdir qualityfirstsoftware/qftest-interactive -batch /qftest/qftest-container/demo/carconfigSwing/carconfigSwing_en.qft

# after the test, the runlog should be accessible from the host
ls -1 container_workdir

You can also use the bound workdir for the reverse operation, i.e. populating the test environment with additional files (e.g. custom test suites).

⁠Interactive usage

The produced runlog can be inspected from the host site, or, an interactive container session can be used. For this, we again bind the container_workdir (containing the runlog) into the container. Recall, that the runlog is located directly in the working directory on the container site. For interactive usage (implies that no --batch option is passed to QF-Test), the VNC server port (5901) has to be published (in this case also to port 5901), so that it can be reached from the outside (--publish|-p option)

docker run -p 5901:5901 -v $(pwd)/container_workdir:/workdir qualityfirstsoftware/qftest-interactive carconfigSwing_en.qrz

Finally stop the container session by simply closing the QF-Test application within the interactive desktop.

⁠Passing the license

If you want to run self-written test suites, QF-Test requires a valid license. It can be passed into the container by the previously introduced bind mount technique. Within the container, the license should be located at: /qftest/qftest-container/license. You can request a trial license here: https://www.qfs.de/en/free-trial.html⁠

# replace <path/to/license> with the actual location of the license on the host system
docker run -p 5901:5901 -v $(pwd)/container_workdir:/workdir -v <path/to/license>:/qftest/qftest-container/license qualityfirstsoftware/qftest-interactive

The license enables you to to save your custom test suites. With the given setup a suite stored in the container's /workdir will also be available on the host site (within the container_workdir) and persist container cycles.

⁠Tweaks and settings

⁠Interactive Container Environment

• The default timezone of the container is UTC. The startup routine will consult the TZ environment variable and reconfigure the container system accordingly. E.g. for Central European Time you could set ENV TZ=Europe/Berlin.
• Without adjustment, the QF-Test instance serves as the main process of the container. (In the interactive environment, it will be be started after VNC server and desktop have been set up.) This process can be changed by means of the MAINPROCESS variable, which shall point to an executable program. This feature might be handy if you have your own wrapping/management script that in turn calls QF-Test.
• The container will be run with /workdir as initial working directory. Use it to equip your test environment with test suites to be run, binaries, etc. Runlogs will also appear here. Exchange data with the test container by mounting a volume/persistent storage under /workdir.

⁠QF-Test execution

• Within the container, the given QF-Test version is installed at /qftest/qftest-container.
• QF-Test will look for a license at /qftest/qftest-container/license. If needed, bind mount a valid license file at this path.
• Leaving the MAINPROCESS variable unchanged implies that all command line arguments given to the container will be forwarded to the qftest process. An extensive list of valid command line arguments is given in the QF-Test manual⁠.
• The return value of a docker run (qftest) command delivers basic information about the test outcome as specified here⁠.

⁠VNC Server

• The VNC server is controllable via the environment variables GEOMETRY (size of the rendered desktop, default: 1680x1050) and VNCPASSWD (default: qf-test). Internally, it listens on port :5901 for incoming connections.

⁠Chrome Crashes

A Docker container runs with 64Mb shared memory by default. This can lead to problems when using Chrome, as it requires more shared memory depending on the web application loaded and can otherwise crash. You can work around this by giving the container more shared memory at startup via:

docker run --shm-size=256m ...

⁠Use cases of the minimal container

⁠Run on external Xserver

The minimal container needs an active Xserver to draw its applications on. Feed the host X-server into the container -- GUI applications will appear on the given DISPLAY.

⁠Local Xserver

If your docker host system offers a running Xserver you can use the minimal QF-Test container and render all GUI windows on the local DESKTOP:

# First, give the root user access to the running X-server:
xhost +local:root

# Run your command
docker run -v ~/.qftest/license:/qftest/license -v /tmp/.X11-unix:/tmp/.X11-unix --env DISPLAY qualityfirstsoftware/qftest-minimal

# Protect the X-server again
xhost -local:root

⁠Headless chrome

When using chrome headless mode (and QF-Test itself in batch mode), web tests with the qftest-minimal-chrome image can be run without a linked X-server. The following command will run a demo web test suite in a headless chrome browser, test results will appear in the bound directory sutdir on the docker host:

docker run -v $(pwd)/sutdir:/workdir qualityfirstsoftware/qftest-minimal-chrome -batch -variable browsername=headless-chrome /qftest/qftest-container/demo/carconfigWeb/carconfigWeb_en.qft