Public | Automated Build

Last pushed: 3 months ago
Short Description
Adds some tools to quay.io/fenicsproject/dev. See https://bitbucket.org/mdbenito/fenics-dev-docker
Full Description

Fenics Docker container

The Dockerfile in this repo adds sshd and a couple of tools to
quay.io/fenicsproject/dev:latest.
This is the base development image containing all the dependencies and
the sources. To obtain an image without the complete code, modify the
Dockerfile to derive from
quay.io/fenicsproject/dev-env:latest.

See:

There are three ways of installing the sources:

  1. Into the docker image. Every container started from this image will
    use the version of the sources in the image. Of course one can pull
    any new commits. This is currently not done in the Dockerfile, but
    is the default in the official FEniCS docker image
    dev:latest. See below for some info on using git within the
    container.
  2. Into a running container. After removing it (docker rm fenics-dev), the code is lost. You can install the sources running
    /home/fenics/bin/fenics-pull. See below for some info on using
    git within the container.
  3. Into a volume shared between the host and the containers.
    Warning: this is slow under MacOS!

For the first two use cases, we can use ssh to edit the files,
e.g. with emacs + tramp. For the third case we can just edit the files
locally. Note however that any IDE running in the host machine
will most likely be confused (missing dependencies, etc.).

As mentioned, we can use ssh (or sshfs) to edit the files or any
editor which is "docker-aware", like PyCharm or emacs
with
TRAMP method for Docker.
These use docker exec to start a shell in the container and does all
the communication through it without ssh.

I'm not sure about dropping ssh altogether, though, since tunneling
ports to a running container with ssh is quite handy (e.g. with
anaconda-mode).

Building the image and setting up SSH

Be sure to copy your id_rsa.pub to this directory before building:

docker build -t fenics-dev .

This will build the image based on the latest official development
container from quay.io. Note that we have given the container the name
fenics-dev.

Also add the following connection alias to your ~/.ssh/config. This
will allow you to access the container via ssh and emacs + tramp easily.

Host fenics
    User fenics
    Hostname localhost
    Port 52022

Running the container

Create or go to some directory in your host machine which you will
want to share with the container, then start it with

docker run -d \
           -v $(pwd):/home/fenics/shared \
           -p 127.0.0.1:8888:8888 \
           -p 127.0.0.1:9000:9000 \
           -p 127.0.0.1:52022:22 \
           --name fenics-dev \
           --hostname fenics-dev \
           --security-opt seccomp=unconfined \
           fenics-dev

In this command we set up port redirections for jupyter notebooks
(8888), ssh (22) and anaconda-mode (9000). See below. The security-opt
argument removes all restrictions in the container. In particular it
makes it possible to use breakpoints in gdb for debugging. See
the docker docs for more info.

If, on the contrary, you'd rather keep the source files in the host
machine, then replace PATH_TO_SOURCES with the path to the directory
containing all of fenics' core projects in the following command:

docker run -d \
           -v $(pwd):/home/fenics/shared \
           -v PATH_TO_SOURCES:/home/fenics/local/src \
           -p 127.0.0.1:8888:8888 \
           -p 52022:22 \
           -p 9000:9000 \
           --name fenics-dev \
           --hostname fenics-dev \
           --security-opt seccomp=unconfined \
           fenics-dev

To open a shell inside the running container use:

docker run -it -u fenics fenics-dev /bin/bash

Starting a notebook

From the host machine (note the -d to run the command in detached
mode, this will be a problem if token authentication is enabled in
jupyter (by default in newer versions). If so, use -it instead to
see the token. Alternatively, add a password to jupyter, etc.):

docker exec -d -u fenics fenics-dev /home/fenics/bin/fenics-notebook

Alternatively, from within the container, as user fenics:

cd ~/shared
jupyter-notebook --no-browser --ip=0.0.0.0

Using git

Inside the container, use fenics-pull to pull the latest changes
from upstream. The commands fenics-build or `fenics-build

<module-name>build and install into~/local/lib`.

If you set up your own remote and want to commit remember to set your
git identity in the container with

git config --global user.name "Mr. You"
git config --global user.email "you@your.server"

You will probably also want to commit the docker image in order not to
loose stuff if you remove the container you are working on or if you
want to start several. See docker commit for that.

Python with emacs + tramp + ssh/docker + anaconda-mode

If you placed the sources inside the container there are two ways to
edit files from the host machine:

  1. tramp + ssh
  2. tramp + docker-tramp

tramp + ssh

Select ssh as the default method in your emacs init with

(setq tramp-default-method "ssh")

Then start the service in the container with:

docker exec -it fenics-dev /usr/sbin/service ssh start

Now within emacs at the host: C-x C-f /fenics: RET opens files
remotely using tramp (if you modified ~/.ssh/config as described above).

tramp + docker

Simply install docker-tramp from melpa with M-x package-install
and do C-x C-f /docker:fenics@fenics-dev:/home/fenics/shared/some-file.py.

If you wish to omit the docker: at the beginning you may select
docker as the default method in your emacs init with

(setq tramp-default-method "docker")

anaconda-mode

To configure anaconda-mode
for fenics you first need to:

  • Add fenics to /etc/hosts in the host machine in order for
    anaconda-mode to connect to the container using the alias we defined
    before. There might be a better way but this does the trick.
  • If you didn't add port 9000 to the command line starting the image,
    forward the default port for anaconda-mode from the host to the
    container: ssh -L 9000:localhost:9000 fenics -f -N. Note: If
    the anaconda-mode process which will run in the container is
    respawned for some reason, the port is likely to change, so you'll
    need to set up new tunnels, or fix the port by modifying the server
    code in
    /home/fenics/.emacs.d/anaconda-mode/<version>/anaconda_mode-<version>-<python>.egg/

Inside emacs, visit some remote python buffer, then:

  • Set the remote interpreter: M-: (setq python-shell-interpreter "/ssh:fenics:/usr/bin/python") NOTE: This is wrong if we want
    to use run-python to open a shell while visiting a tramp buffer
    because tramp will already try to run the interpreter remotely, so
    we need to (setq python-shell-interpreter "/usr/bin/python"),
    which will cause anaconda-mode to fail, or use instead M-: (run-python "/usr/bin/python").
  • One can / must also set the python environment ... document this
  • Enjoy / suffer the bugs :P

Debugging python in the container with realgud:pdb

  1. Open some file in the container, e.g. C-x C-f /docker:fenics@fenics-dev:/home/fenics/shared/test.py
  2. Run realgud:pdb with the remote path as if it were local, i.e. pdb /home/fenics/shared/test.py. If you see ImportError: 'No module named _common' when importing dolfin, you need to setup the
    environment variables first by sourcing
    /home/fenics/fenics.env.conf before calling pdb. Note:
    This is already done in /home/fenics/bin/pdb, so the command to
    start a debugging session on the current buffer is s-d /home/fenics/bin/pdb <filename>.

Bugs

  • realgud sometimes (always?) fails to convert remote filenames to
    tramp filenames (i.e /home/fenics/blah to
    /docker:fenics@fenics-dev:/home/fenics/blah)

  • Is this true? realgud seems to be confused by some replies from
    pdb: when stepping into a function a line with the content
    --Call-- is prepended to the output and this confuses the regex
    parsing the filename. A question pops up in the minibuffer for the
    file. This could be cached in some way because the error isn't
    consistent (?). I tried rewriting the regex in
    realgud/debugger/pdb/init.el, but it's probably wrong or not
    supposed to be multiline or whatever:

(setf (gethash "loc" realgud:pdb-pat-hash)
      (make-realgud-loc-pat
       :regexp "^\\(?:--Call--
\\|--Return--
\\)*> \\(\\(?:[a-zA-Z]:\\)?[-a-zA-Z0-9_/.\\\\ ]+\\)(\\([0-9]+\\))"
       :file-group 1
       :line-group 2))

Probably the same happens upon stepping out of a function, because the
debugger appends --Return--.

As a workaround I edited docker:fenics@fenics-dev:/usr/bin/pdb not
to output that stuff.

C++ with emacs + tramp + rtags / cmake-ide

rtags indexes C++ code and
stores the results in a database. It has an emacs mode and can be
used for completion, jumping to definitions etc. It should already be
in the docker container.

cmake-ide should take care
of running cmake and rtags, as well as setting up flycheck,
clang-complete and whatnot.

rtags

To use it you need to:

  1. Start the server in the container:

    rdm --default-argument -fopenmp=libomp --daemon

    We need to add libomp manually since it is disabled by default in
    clang. Add -Q to disable reindexing upon startup, useful if one
    is testing and restarting often.

  2. Add set(CMAKE_EXPORT_COMPILE_COMMANDS ON) to the CMakeLists.txt
    of the projects you will work with, then execute cmake. This
    produces a file compile_commands.json in the build
    directory. Alternatively, run

    cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ..

    from the build directory of the project

  3. Use the client and the compile commands from above to parse the
    code: rc -J [build-directory].
  4. NOTE: It is very important that the rtags elisp file in your
    host's emacs is at the same version than the rc client in the
    container. Remember to rebuild rtags whenever you update the emacs
    package. The version is fixed in the Dockerfile to v2.9 as of
    4.2017.

Useful options for rc while tinkering with the setup are: rc -q to
kill the server, rc -w to list the known projects and rc -C to
clear all projects. Finally, if you need to reset the database stop
the server and run it again with --clean-slate.

Indexing will take a while. You can setup emacs in the meantime:

  1. Add something like this to your config:

    (require 'rtags)
    (setq rtags-tramp-enabled t)
    (setq rtags-autostart-diagnostics t)
    (setq rtags-completions-enabled t)
    (require 'company)
    (global-company-mode)
    (add-to-list 'company-backends 'company-rtags)
    
  2. Add any shortcuts, e.g. for rtags-find-symbol-at-point.

cmake-ide-mode

To do: There are issues running over tramp. Until I prepare a PR for
cmake-ide, using it via tramp might require some tweaking of
cmake-ide.el, in particular replacing some function calls which work
only for local buffers by some others which base their actions on
default-directory and hence automagically support tramp buffers. The
substitutions to make are (with no change in parameters):

local only local and remote
call-process process-file
start-process start-file-process
... ...

However more changes are required in order for cmake-ide to really
work. I will commit them to my fork of cmake-ide at some point...

See
this issue in irony-mode for
more.

Also, usage of .dir-locals.el in each subproject is necessary for
cmake-ide to know where to find things. Note in the following example
that paths should be local:

((nil .
      ((compile-command . "cd build && make -k -j4")
       (cmake-ide-build-dir . "/home/fenics/local/src/nonlinear-kirchhoff/build")
       (c-basic-offset . 2)
       (c-default-style . "linux"))))

Directory local variables are disabled by default in emacs. You can
enable them with

(setq enable-remote-dir-locals t)

More emacs hints

There is a fenics-minor-mode
here
which uses a file local variable with the fenics dependencies of a file
in order to rebuild the necessary fenics components before
debugging. This is only useful for a particular workflow of mine,
which helped reverse engineering / hacking / brute-forcing my way into
implementing Hermite and Kirchhoff elements.

License

Released under the GPL v3.

Docker Pull Command
Owner
mdbenito
Source Repository

Comments (0)