Doodba

Doodba stands for Docker Odoo Base, and it is a highly opinionated image ready to put Odoo⁠ inside it, but without Odoo.

What?

Yes, the purpose of this is to serve as a base for you to build your own Odoo project, because most of them end up requiring a big amount of custom patches, merges, repositories, etc. With this image, you have a collection of good practices and tools to enable your team to have a standard Odoo project structure.

BTW, we use Debian⁠. I hope you like that.

Why?

Because developing Odoo is hard. You need lots of customizations, dependencies, and if you want to move from one version to another, it's a pain.

Also because nobody wants Odoo as it comes from upstream, you most likely will need to add custom patches and addons, at least, so we need a way to put all together and make it work anywhere quickly.

How?

You can start working with this straight away with our [scaffolding][].

Image usage

Basically, every directory you have to worry about is found inside /opt/odoo. This is its structure:

custom/
    entrypoint.d/
    build.d/
    conf.d/
    ssh/
        config
        known_hosts
        id_rsa
        id_rsa.pub
    dependencies/
        apt_build.txt
        apt.txt
        gem.txt
        npm.txt
        pip.txt
    src/
        private/
        odoo/
        addons.yaml
        repos.yaml
common/
    entrypoint.sh
    build.sh
    entrypoint.d/
    build.d/
    conf.d/
auto
    addons/
    odoo.conf

Let's go one by one.

/opt/odoo/custom: The important one

Here you will put everything related to your project.

/opt/odoo/custom/entrypoint.d

Any executables found here will be run when you launch your container, before running the command you ask.

/opt/odoo/custom/build.d

Executables here will be aggregated with those in /opt/odoo/common/build.d.

The resulting set of executables will then be sorted alphabetically (ascending) and then subsequently run.

/opt/odoo/custom/conf.d

Files here will be environment-variable-expanded and concatenated in /opt/odoo/auto/odoo.conf in the entrypoint.

/opt/odoo/custom/ssh

It must follow the same structure as a standard ~/.ssh directory, including config and known_hosts files. In fact, it is completely equivalent to ~root/.ssh.

The config file can contain IdentityFile keys to represent the private key that should be used for that host. Unless specified otherwise, this defaults to identity[.pub], id_rsa[.pub] or id_dsa[.pub] files found in this same directory.

This is very useful to use deployment keys that grant git access to your private repositories.

Example - a private key file in the ssh folder named my_private_key for the host repo.example.com would have a config entry similar to the below:

Host repo.example.com
  IdentityFile ~/.ssh/my_private_key

Or you could just drop the key in id_rsa and id_rsa.pub files and it should work by default without the need of adding a config file.

Host key checking is enabled by default, which means that you also need to provide a known_hosts file for any repos that you wish to access via SSH.

In order to disable host key checks for a repo, your config would look something like this:

Host repo.example.com
  StrictHostKeyChecking no

For additional information regarding this directory, take a look at this [Digital Ocean Article][ssh-conf].

/opt/odoo/custom/src

Here you will put the actual source code for your project.

When putting code here, you can either:

• Use [repos.yaml][], that will fill anything at build time.
• Directly copy all there.

Recommendation: use [repos.yaml][] for everything except for [private][], and ignore in your .gitignore and .dockerignore files every folder here except [private][], with rules like these:

odoo/custom/src/*
!odoo/custom/src/private
!odoo/custom/src/*.*

/opt/odoo/custom/src/odoo

REQUIRED. The source code for your odoo project.

You can choose your Odoo version, and even merge PRs from many of them using [repos.yaml][]. Some versions you might consider:

• [Original Odoo][], by [Odoo S.A.][].

• [OCB][] (Odoo Community Backports), by [OCA][]. The original + some features - some stability strictness.

• [OpenUpgrade][], by [OCA][]. The original, frozen at new version launch time + migration scripts.

/opt/odoo/custom/src/private

REQUIRED. Folder with private addons for the project.

/opt/odoo/custom/src/repos.yaml

A git-aggregator configuration file.

It should look similar to this:

# Odoo must be in the `odoo` folder for Doodba to work
odoo:
  defaults:
    # This will use git shallow clones.
    # $DEPTH_DEFAULT is 1 in test and prod, but 100 in devel.
    # $DEPTH_MERGE is always 100.
    # You can use any integer value, OTOH.
    depth: $DEPTH_MERGE
  remotes:
    origin: https://github.com/OCA/OCB.git
    odoo: https://github.com/odoo/odoo.git
    openupgrade: https://github.com/OCA/OpenUpgrade.git
  # $ODOO_VERSION is... the Odoo version! "11.0" or similar
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION
    - odoo refs/pull/25594/head # Expose `Field` from search_filters.js

web:
  defaults:
    depth: $DEPTH_MERGE
  remotes:
    origin: https://github.com/OCA/web.git
    tecnativa: https://github.com/Tecnativa/partner-contact.git
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION
    - origin refs/pull/1007/head # web_responsive search
    - tecnativa 11.0-some_addon-custom # Branch for this customer only

Automatic download of repos

Doodba is smart enough to download automatically git repositories even if they are missing in repos.yaml. It will happen if it is used in [addons.yaml][], except for the special [private][] repo. This will help you keep your deployment definitions DRY.

You can configure this behavior with these environment variables (default values shown):

• DEFAULT_REPO_PATTERN="https://github.com/OCA/{}.git"
• DEFAULT_REPO_PATTERN_ODOO="https://github.com/OCA/OCB.git"

As you probably guessed, we use something like str.format(repo_basename) on top of those variables to compute the default remote origin. If, i.e., you want to use your own repositories as default remotes, just add these build arguments to your docker-compose.yaml file:

# [...]
services:
  odoo:
    build:
      args:
        DEFAULT_REPO_PATTERN: &origin "https://github.com/Tecnativa/{}.git"
        DEFAULT_REPO_PATTERN_ODOO: *origin
# [...]

So, for example, if your [repos.yaml][] file is empty and your [addons.yaml][] contains this:

server-tools:
- module_auto_update

A /opt/odoo/auto/repos.yaml file with this will be generated and used to download git code:

/opt/odoo/custom/src/odoo:
  depth: $DEPTH_DEFAULT
  remotes:
    origin: https://github.com/OCA/OCB.git
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION
/opt/odoo/custom/src/server-tools:
  depth: $DEPTH_DEFAULT
  remotes:
    origin: https://github.com/OCA/server-tools.git
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION

All of this means that, you only need to define the git aggregator spec in [repos.yaml][] if anything diverges from the standard:

• You need special merges.
• You need a special origin.
• The folder name does not match the origin pattern.
• The branch name does not match $ODOO_VERSION.
• Etc.

/opt/odoo/custom/src/addons.yaml

One entry per repo and addon you want to activate in your project. Like this:

website:
    - website_cookie_notice
    - website_legal_page
web:
    - web_responsive

Advanced features:

• You can bundle [several YAML documents][] if you want to logically group your addons and some repos are repeated among groups, by separating each document with ---.

• Addons under private and odoo/addons are linked automatically unless you specify them.

• You can use ONLY to supply a dictionary of environment variables and a list of possible values to enable that document in the matching environments.

• If an addon is found in several places at the same time, it will get linked according to this priority table:

  1. Addons in [private][].
  2. Addons in other repositories (in case one is matched in several, it will be random, BEWARE!). Better have no duplicated names if possible.
  3. Core Odoo addons from [odoo/addons][odoo].

• If an addon is specified but not available at runtime, it will fail silently.

• You can use any wildcards supported by [Python's glob module][glob].

This example shows these advanced features:

# Spanish Localization
l10n-spain:
  - l10n_es # Overrides built-in l10n_es under odoo/addons
server-tools:
  - "*date*" # All modules that contain "date" in their name
  - auditlog
web:
  - "*" # All web addons
---
# Different YAML document to separate SEO Tools
website:
  - website_blog_excertp_img
server-tools: # Here we repeat server-tools, but no problem because it's a
              # different document
  - html_image_url_extractor
  - html_text
---
# Enable demo ribbon only for devel and test environments
ONLY:
  PGDATABASE: # This environment variable must exist and be in the list
    - devel
    - test
web:
  - web_environment_ribbon
---
# Enable special authentication methods only in production environment
ONLY:
  PGDATABASE:
    - prod
server-tools:
  - auth_*

/opt/odoo/custom/dependencies/*.txt

Files to indicate dependencies of your subimage, one for each of the supported package managers:

• apt_build.txt: build-time dependencies, installed before any others and removed after all the others too. Usually these would include Debian packages such as build-essential or python-dev. From Doodba 11.0, this is most likely not needed, as build dependencies are shipped with the image, and local python develpment headers should be used instead of those downloaded from apt.
• apt.txt: run-time dependencies installed by apt.
• gem.txt: run-time dependencies installed by gem.
• npm.txt: run-time dependencies installed by npm.
• pip.txt: a normal [pip requirements.txt][] file, for run-time dependencies too. It will get executed with --update flag, just in case you want to overwrite any of the pre-bundled dependencies.

/opt/odoo/common: The useful one

This folder is full of magic. I'll document it some day. For now, just look at the code.

Only some notes:

• Will compile your code with [PYTHONOPTIMIZE=1][] by default.

• Will remove all code not used from the image by default (not listed in /opt/odoo/custom/src/addons.yaml), to keep it thin.

/opt/odoo/auto: The automatic one

This directory will have things that are automatically generated at build time.

/opt/odoo/auto/addons

It will be full of symlinks to the addons you selected in [addons.yaml][].

/opt/odoo/auto/odoo.conf

It will have the result of merging all configurations under /opt/odoo/{common,custom}/conf.d/, in that order.

The Dockerfile

I will document all build arguments and environment variables some day, but for now keep this in mind:

• This is just a base image, full of tools. You need to build your project subimage from this one, even if your project's Dockerfile only contains these 2 lines:

  FROM tecnativa/doodba
  MAINTAINER Me <me@example.com>
  

  Copy

• The above sentence becomes true because we have a lot of ONBUILD sentences here, so at least your project must have a ./custom folder along with its Dockerfile for it to work.

• All should be magic if you adhere to our opinions here. Just put the code where it should go, and relax.

Bundled tools

There is a good collections of tools available in the image that help dealing with Odoo and its peculiarities:

addons

A handy CLI tool to automate addon management based on the current environment. It allows you to install, update, test and/or list private, extra and/or core addons available to current container, based on current [addons.yaml][] configuration.

Call addons --help for usage instructions.

click-odoo and related scripts

The great [click-odoo][] scripting framework and the collection of scripts found in [click-odoo-contrib][] are included. Refer to their sites to know how to use them.

* Note: This replaces the deprecated python-odoo-shell binary.

[nano][]

The CLI text editor we all know, just in case you need to inspect some bug in hot deployments.

log

Just a little shell script that you can use to add logs to your build or entrypoint scripts:

log INFO I'm informing

pot

Little shell shortcut for exporting a translation template from any addon(s). Usage:

pot my_addon,my_other_addon

psql⁠

Environment variables are there so that if you need to connect with the database, you just need to execute:

docker exec -it your_container psql

The same is true for any other [Postgres client applications][].

ptvsd⁠

[VSCode][] debugger. If you use this editor with its python module, you will find it useful.

To debug at a certain point of the code, add this Python code somewhere:

import ptvsd
ptvsd.enable_attach("doodba-rocks", address=("0.0.0.0", 6899))
print("ptvsd waiting...")
ptvsd.wait_for_attach()

To start Odoo within a ptvsd environment, which will obey the breakpoints established in your IDE (but will work slowly), just add -e PTVSD_ENABLE=1 to your odoo container.

If you use the official [scaffolding][], you can boot it in ptvsd mode with:

export DOODBA_PTVSD_ENABLE=1
docker-compose -f devel.yaml up -d

Of course, you need to have properly configured your [VSCode][]. To do so, make sure in your project there is a .vscode/launch.json file with these minimal contents:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Attach to debug in devel.yaml",
            "type": "python",
            "request": "attach",
            "pathMappings": [
                {
                    "localRoot": "${workspaceRoot}/odoo",
                    "remoteRoot": "/opt/odoo"
                }
            ],
            "port": 6899,
            "host": "localhost"
        }
    ]
}

Then, execute that configuration as usual.

pudb⁠

This is another great debugger that includes remote debugging via telnet, which can be useful for some cases, or for people that prefer it over wdb.

To use it, inject this in any Python script:

import pudb.remote
pudb.remote.set_trace(term_size=(80, 24))

Then open a telnet connection to it (running in 0.0.0.0:6899 by default).

It is safe to use in [production][] environments if you know what you are doing and do not expose the debugging port to attackers. Usage:

docker-compose exec odoo telnet localhost 6899

git-aggregator⁠

We found this one to be the most useful tool for downlading code, merging it and placing it somewhere.

autoaggregate

This little script wraps git-aggregator to make it work fine and automatically with this image. Used in the [scaffolding][]'s setup-devel.yaml step.

Example [repos.yaml][] file

This example merges [several sources][odoo]:

./odoo:
    defaults:
        # Shallow repositores are faster & thinner. You better use
        # $DEPTH_DEFAULT here when you need no merges.
        depth: $DEPTH_MERGE
    remotes:
        ocb: https://github.com/OCA/OCB.git
        odoo: https://github.com/odoo/odoo.git
    target:
        ocb $ODOO_VERSION
    merges:
        - ocb $ODOO_VERSION
        - odoo refs/pull/13635/head

odoo⁠

We set an $OPENERP_SERVER environment variable pointing to the autogenerated configuration file so you don't have to worry about it. Just execute odoo and it will work fine.

Note that version 9.0 has an odoo binary to provide forward compatibility (but it has the odoo.py one too).

Scaffolding

Get up and running quickly with the provided scaffolding⁠.

Skip the boring parts

You will need these tools, so install them locally (and learn how to use them, check their docs, Doodba is not the place to learn them 😉):

• Git⁠
• Docker Engine⁠ (running locally)
• Docker Compose⁠

Then run these Bash commands:

git clone https://github.com/Tecnativa/doodba-scaffolding.git myproject
cd myproject
ln -s devel.yaml docker-compose.yml
chmod -R ug+rwX odoo/auto
export UID GID="$(id -g $USER)" UMASK="$(umask)"
docker-compose build --pull
docker-compose -f setup-devel.yaml run --rm odoo
docker-compose up

And if you don't want to have a chance to do a git pull and get possible future scaffolding updates merged in your project's git log:

rm -Rf .git
git init

Tell me the boring parts

The scaffolding provides you a boilerplate-ready project to start developing Odoo in no time.

Environments

This scaffolding comes with some environment configurations, ready for you to extend them. Each of them is a Docker Compose file⁠ almost ready to work out of the box (or almost), but that will assume that you understand it and will modify it.

After you clone the scaffolding, search for XXX comments, they will help you on making it work.

Development

Set it up with:

export UID GID="$(id -g $USER)" UMASK="$(umask)"
docker-compose -f setup-devel.yaml run --rm odoo

Once finished, you can start using Odoo with:

docker-compose -f devel.yaml up --build

This allows you to track only what Git needs to track and provides faster Docker builds.

You might consider adding this line to your ~/.bashrc:

export UID GID="$(id -g $USER)" UMASK="$(umask)"

To browse Odoo go to http://localhost:${ODOO_MAJOR}069 (i.e. for Odoo 11.0 this would be http://localhost:11069).

This environment has several special features:

wdb⁠

This is one of the greatest Python debugger available, and even more for Docker-based development, so here you have it preinstalled.

I told you, this image is opinionated. :wink:

To use it, write this in any Python script:

import wdb
wdb.set_trace()

It's available by default on the [development][] environment, where you can browse http://localhost:1984⁠ to use it.

DO NOT USE IT IN PRODUCTION ENVIRONMENTS. (I had to say it).

MailHog⁠

It provides a fake SMTP server that intercepts all mail sent by Odoo and displays a simple interface that lets you see and debug all that mail comfortably, including headers sent, attachments, etc.

• For [development][], it's in http://localhost:8025⁠
• For [testing][], it's in http://$DOMAIN_TEST/smtpfake/
• For [production][], it's not used.

All environments are configured by default to use the bundled SMTP relay. They are configured by these environment variables:

• SMTP_SERVER
• SMTP_PORT
• SMTP_USER
• SMTP_PASSWORD
• SMTP_SSL
• EMAIL_FROM

For them to be useful, you need to remove any ir.mail_server records in your database.

Network isolation

The Docker network is in --internal mode, which means that it has no access to the Internet. This feature protects you in cases where a [production][] database is restored and Odoo tries to connect to SMTP/IMAP/POP3 servers to send or receive emails. Also when you are using connectors⁠, mail trackers⁠ or any API sync/calls.

If you still need to have public access, set internal: false in the environment file, detach all containers from that network, remove the network, reatach all containers to it, and possibly restart them. You can also just do:

docker-compose down
docker-compose up -d

Usually a better option is whitelisting.

Production

This environment is just a template. It is not production-ready. You must change many things inside it, it's just a guideline.

It includes pluggable smtp and backup services.

Adding secrets

Before booting this environment, you need to create a few files, which are excluded in Git and contain some secrets, needed to make this environment safe:

• ./.docker/odoo.env must define ADMIN_PASSWORD.
• ./.docker/db-access.env must define PGPASSWORD.
• ./.docker/db-creation.env must define POSTGRES_PASSWORD (must be equal to PGPASSWORD above).
• ./.docker/smtp.env must define MAIL_RELAY_PASS (password to access the real SMTP relay).
• ./.docker/backup.env must define AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY (obtained from S3 provider) and PASSPHRASE (to encrypt backup archives).

Booting production

Once you fixed everything needed, run it with:

docker-compose -f prod.yaml up --build --remove-orphans

Global inverse proxy

For [production][] and [test][] templates to work fine, you need to have a working [Traefik][] inverse proxy in each node.

To have it, use this inverseproxy.yaml file:

version: "2.1"

services:
    proxy:
        image: traefik:1.6-alpine
        networks:
            shared:
            private:
            public:
        volumes:
            - acme:/etc/traefik/acme:rw,Z
        ports:
            - "80:80"
            - "443:443"
        depends_on:
            - dockersocket
        restart: unless-stopped
        privileged: true
        tty: true
        command:
            - --ACME.ACMELogging
            - --ACME.Email=you@example.com
            - --ACME.EntryPoint=https
            - --ACME.HTTPChallenge.entryPoint=http
            - --ACME.OnHostRule
            - --ACME.Storage=/etc/traefik/acme/acme.json
            - --DefaultEntryPoints=http,https
            - --EntryPoints=Name:http Address::80 Redirect.EntryPoint:https
            - --EntryPoints=Name:https Address::443 TLS
            - --LogLevel=INFO
            - --Docker
            - --Docker.EndPoint=http://dockersocket:2375
            - --Docker.ExposedByDefault=false
            - --Docker.Watch

    dockersocket:
        image: tecnativa/docker-socket-proxy
        privileged: true
        networks:
            private:
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
        environment:
            CONTAINERS: 1
            NETWORKS: 1
            SERVICES: 1
            SWARM: 1
            TASKS: 1
        restart: unless-stopped

networks:
    shared:
        internal: true
        driver_opts:
            encrypted: 1

    private:
        internal: true
        driver_opts:
            encrypted: 1

    public:

volumes:
    acme:

Then boot it up with:

docker-compose -p inverseproxy -f inverseproxy.yaml up -d

This will intercept all requests coming from port 80 (http) and redirect them to port 443 (https), it will download and install required SSL certificates from [Let's Encrypt][] whenever you boot a new [production][] instance, add the required proxy headers to the request, and then redirect all traffic to/from odoo automatically.

It includes [a security-enhaced proxy][docker-socket-proxy] to reduce attack surface when listening to the Docker socket.

This allows you to:

• Have multiple dom