LHCbPR2 Developers Package
<!-- TOC -->
- LHCbPR2 Developers Package
<!-- /TOC -->
- Git — to get the lhcbpr2 source
- Docker — to run applications in the same development environment for every user
- Docker Compose — defining and running multi-container Docker applications.
Docker containers wrap a piece of software in a complete filesystem that
contains everything needed to run: code, runtime, system tools, system
libraries – anything that can be installed on a server.
This guarantees that the software will always run the same,
regardless of its environment.
- Docker ≥ 1.10
# Check docker version $ docker --version # Docker version 1.12.3-rc1, build bad4d12, experimental
At linux, It's recommended to add your user account to the "docker" group, so you can run
docker containers without root rights.
$ sudo groupadd docker $ sudo usermod -aG docker $USER
Don't forget to verify that you can run docker:
$ docker run hello-world
If this fails with a message similar to this:
Cannot connect to the Docker daemon. Is 'docker daemon' running on this host?
Compose is a tool for defining and running multi-container Docker applications.
Compose is preinstalled if you use Docker for Mac or Windows
- docker-compose ≥ 1.8.1
# Check docker-compose version $ docker-compose --version # docker-compose version 1.8.1, build 878cff1
First, you need to clone this project:
$> git clone https://gitlab.cern.ch/amazurov/LHCbPR2.git
, and run bootstrap script
$> cd LHCbPR2 $> ./scripts/bootstrap
, which clone the following subprojects into the
- LHCbPR2BE - API backend server
- LHCbPR2FE - Web frontend for API server
- LHCbPR2ROOT - Utility server for retreiving information from root files
- LHCbPR2HD - Prepare jobs output for import into LHCbPR
- LbNightlyTools - Run tests and call handlers from LHCbPR2HD
Run All Services
Run all services in the development environment
docker-compose up -d
, runs development environment from
docker-compose.yml configuration. First run of the command
can take some time.
Run all services in the other environment [optional]
You can select another environment, by using the following command:
docker-compose -f <docker-compose-file.yml> -p <project-name> up -d
, where docker-compose-file.yml and <project-name> can be:
lhcbpr2dev- run subprojects in development mode:
- All changes in the subproject's code immidiately applyed
- Runs subprojects' internal web servers for debug purposes
lhcbpr2prod- run subprojects in mode close to production
- Runs subprojects' services in production web server: apache, nginx or gunicorn.
USEFUL: You can avoid adding
-p options by creating the
.env file in the root of the project with the following values:
(change left values to what you need)
In the instruction bellow I will ommit -f and -p options for the docker-compose command.
IMPORTANT: if you docker machine url is not
localhost (it's used by default) then change the
in the corresponding compose configuration (by default the confuguration file is
lhcbpr2all service, e.g:
... lhcbpr2all: ... environment: APP_HOST: your_host_url ... ports: ...
After running the command all services should be started and have up state (except lhcbpr2night service):
Name Command State Ports ------------------------------------------------------------------------------------------------------------------------------ lhcbpr2_lhcbpr2all_1 /usr/local/bin/ep -v /etc/ ... Up 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp lhcbpr2_lhcbpr2be_1 ./scripts/runserver Up 0.0.0.0:8082->80/tcp lhcbpr2_lhcbpr2fe_1 ./scripts/runserver Up 0.0.0.0:35729->35729/tcp, 0.0.0.0:8080->80/tcp, 9000/tcp lhcbpr2_lhcbpr2night_1 /usr/bin/cubied bash Exit 0 lhcbpr2_lhcbpr2root_1 ./scripts/bootstrap Up 0.0.0.0:8081->80/tcp
The other status means that something went wrong and you can investigate the problem in logs:
- Global log for all services:
docker-compose logs -f
- Log per service:
docker-compose logs -f <service_name>, e.g.
docker-compose logs -f lhcbpr2fe.
Containers are started when they are in the "up" state (ignore lhcbpr2night container) and application in
the containers are initilized. You can check that applications are initialized when logs does not produce
any output and have no errors. Usually you need to see something like this line after running
docker-compose logs -f:
... lhcbpr2fe_1 | [09:12:52] ************ lhcbpr2fe_1 | [09:12:52] * All Done * You can start editing your code, LiveReload will update your browser after any change.. lhcbpr2fe_1 | [09:12:52] ************ lhcbpr2fe_1 | [09:12:52] Finished 'default' after 842 μs ...
When containers are in the "up" state and application are started in the containers (you can see it when logs does not produce
any output) you can access lhcbpr web site at the following address:
, you need accept unsigned https certificate.
How to run tests without jenkins
Login into lhcbpr2night container with lhcb cvmfs support:
docker-compose run lhcbpr2night bash
In the container:
mkdir /lhcbprdata/output # Or any other directory in /lhcbprdata directory cd /lhcbprdata/output /app/scripts/lbpr-example # Should produce zip file with job results for LHCbPR
At lbpr-example you need to setup your test parameters. Full documentation at LbNightlyTools repository.
The produced zip result you can import to LHCbPR2BE database:
Login into lhcbpr2be container:
docker-compose exec lhcbpr2be bash
./site/manage.py lhcbpr_import /lhcbprdata/output
How to develop analysis modules
How services work in production