Brightspot Local Development Docker Containers

These containers are intended for local development only, and are optimized for this use case. For production use, please contact Brightspot⁠.

1. Download and install Docker Desktop and ensure it is running: https://www.docker.com/products/docker-desktop⁠
2. Create a file in the root of your project called docker-compose.yml with the following contents (replace "${my-project}" with your project name or the appropriate values):

   version: '3'
   services:
     tomcat:
       image: brightspot/tomcat:8-jdk8
       hostname: ${my-project}.brightspot
       ports:
         - 5005:5005
         - 9010:9010
       volumes:
         - .:/code:cached
         - storage-data:/servers/tomcat/storage
         - $HOME/.aws/credentials:/etc/aws/credentials:cached
       environment:
         - ROOT_WAR=/code/site/build/libs/${my-project}-site-1.0.0-SNAPSHOT.war
         - CONTEXT_PROPERTIES=/code/docker-context.properties
         - CONTEXT_PROPERTIES_OVERRIDES=/code/docker-context-overrides.properties
         - LOGGING_PROPERTIES=/code/docker-logging.properties
         - AWS_PROFILE=psd-${my-project}
         - ENABLE_JFR=false
         - SOLR_URL=http://solr:8080/solr/collection1
     solr:
       image: brightspot/solr:8.11.1
       ports:
         - 8180:8080
       volumes:
         - solr-data:/var/solr/data/collection1/data
     apache:
       image: brightspot/apache:2-dims3
       ports:
         - 80:80
         - 443:443
       volumes:
         - storage-data:/var/www/localhost/htdocs/storage
     mysql:
       image: brightspot/mysql:mysql5.6
       volumes:
         - mysql-data:/var/lib/mysql
         - mysql-logs:/var/log/mysql
   volumes:
     storage-data:
     mysql-data:
     mysql-logs:
     solr-data:

• Ensure the relative path to the war file is correct!
• If you plan on restoring MySQL backups using the restore script, make sure the version of MySQL matches the version running in your production environments.

1. Start the containers using docker-compose up or docker-compose up -d. See the docker-compose documentation⁠ for other options.

• NOTE: Only one container per published port can run at the same time. If you want to run two docker containers at the same time, make sure you publish different port numbers (See the Docker documentation⁠ for more details).

1. Access Brightspot at http://localhost/cms⁠

Environment Variables

ROOT_WAR

ROOT_WAR contains the path in the container to the project war file.

CONTEXT_PROPERTIES

CONTEXT_PROPERTIES contains the path to a properties file that will be parsed and added as <Environment> elements in tomcat's context.xml.

For example, docker-context.properties:

dari/debugUsername=debug
dari/debugPassword=12345
dari/debugRealm=Brightspot-Docker

is transformed into context.xml:

   <Environment name="dari/debugUsername" value="debug" type="java.lang.String"/>
   <Environment name="dari/debugPassword" value="12345" type="java.lang.String"/>
   <Environment name="dari/debugRealm" value="Brightspot-Docker" type="java.lang.String"/>

CONTEXT_PROPERTIES_OVERRIDES

CONTEXT_PROPERTIES_OVERRIDES contains the path to a properties file that will be parsed and added after the CONTEXT_PROPERTIES (see above). This allows you to maintain a shared, project-wide configuration and to add local (i.e., not version controlled) values.

Note: If you duplicate values between CONTEXT_PROPERTIES and CONTEXT_PROPERTIES_OVERRIDES, the original value from CONTEXT_PROPERTIES will be replaced with the value in CONTEXT_PROPERTIES_OVERRIDES. If you need to replace the entire configuration, you can alternatively set a different path for CONTEXT_PROPERTIES in your docker-compose.override.yml with the modified version instead.

LOGGING_PROPERTIES

LOGGING_PROPERTIES contains the path to a properties file that will be appended to tomcat's logging.properties.

For example, docker-logging.properties:

com.my.package.level = FINE
com.my.package.MyClass.level = FINE
com.other.package.NoisyClass.level = SEVERE

INIT_SH

INIT_SH contains the path to an initialization script that will be executed when the container starts.

Example:

environment:
  - INIT_SH=/code/docker-tomcat-init.sh

NOTE: Make sure docker-tomcat-init.sh is executable!

AWS_PROFILE

Access to AWS services via your host credentials file is a two-step process:

1. Ensure that your ~/.aws/credentials file is mounted as /etc/aws/credentials in the docker container (See: docker-compose.yml above)

2. Add lines like

   - AWS_PROFILE=psd-${my-project}
   - AWS_REGION=us-east-1

to your docker-compose.yml under environment: and run docker-compose up to apply the change. Note that us-east-1 is already provided as the default region, so you should only have to override this if you need to test with a different region.

Now any changes to your host credentials file (via beam credentials, for example) will be accessible to the DefaultAWSCredentialsProviderChain in the docker container.

ENABLE_JFR

Enables Java Flight Recorder on the Tomcat process.

If Tomcat is started with ENABLE_JFR=true, a JFR dump can be generated using the command:

docker-compose exec tomcat jfr-dump.sh

By default, this will create a file in /code. Use the JFR_DIR environment variable to override this default if desired, or invoke jfr-dump.sh with a filename as the first argument.

The JFR dump file can be analyzed using JDK Mission Control⁠.

TEMPLATE_CONFIG

If TEMPLATE_CONFIG contains the path to a YAML file (for instance TEMPLATE_CONFIG=/code/docker-template.yml), the file will be used to populate the ERB templates listed in /etc/docker/templates.

For example:

data:
  brightspot:
      extra_dari_db: otherdatabase
      extra_resources:
        - name: database1 # Required
          minimum_idle: 4 # Default: 2
          idle_timeout: 20000 # Default: 10000
          maximum_pool_size: 16 # Default: 12
          max_lifetime: 60000 # Default: 30000
          connection_timeout: 2000 # Default: 1000
          host: host # Default: localhost
          port: 3306 # Default: 3306
          database_name: database1 # Default: 'name' property value
          user: user # Required
          password: password # Required
        - name: database2
          minimum_idle: 4
          idle_timeout: 20000
          maximum_pool_size: 16
          max_lifetime: 60000
          connection_timeout: 2000
          host: host
          port: 3306
          database_name: database2
          user: user
          password: password

This populates the @data['brightspot']['extra_dari_db'] setting found in /servers/tomcat/conf/context.xml.erb and /servers/tomcat/conf/server.xml.erb.

It will also create additional, extra resource links in /servers/tomcat/conf/server.xml.erb

Future template configs will be documented here as they are developed.

SSL

SSL is enabled on the Apache server. In order to trust the certificate so any domain will work, run the following commands on your Mac:

curl -O https://psddev.s3.amazonaws.com/brightspot-docker-ca.pem
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain brightspot-docker-ca.pem

Caching

HTTP Request caching is provided via Apache mod_cache⁠ only if Cache-Control headers are found on the response.

To disable mod_cache altogether, add an environment variable to the apache container: DISABLE_CACHE=true

Task Host

The only purpose of the hostname property is to provide a consistent hostname that can be published in site settings as a task host.

Debugger

Remote debugging is exposed on port 5005.

JMX

JMX provides a standard way to monitor Brightspot's performance, resource consumption, and manage various runtime aspects. This is enabled by default.

Connect to localhost:9010 using a tool such as JConsole⁠ or VisualVM⁠.

This port can be customized with the environment variable JMX_PORT. Just make sure you use the same value on both sides in the ports configuration - it must be the same port inside and outside the Docker network.

Local Development (Reloader)

Note: if you are using version 4.1.8 or newer of the Brightspot Gradle Plugin and have added 'systemProp.codePath=/code' to your gradle.properties, this step is unnecessary.

Create a second file docker-compose.override.yml in the root of your project with the following contents:

version: '3'
services:
  tomcat:
    volumes:
      - $PWD:$PWD:cached

This will allow Brightspot to scan the host filesystem to automatically recompile Java source files.

Optional Note for Mac Users

Accessing the host filesystem on Docker for Mac OS has known performance problems⁠. There is a workaround, however, Docker Sync⁠. After installing⁠ Docker Sync (sudo gem install docker-sync), create a file docker-sync.yml in the root of your project with the following contents (replace "${my-project}" with your project name):

version: '2'
syncs:
  ${my-project}-sync:
    src: ./
    sync_excludes: ['node_modules', '.git', '.gradle', '.idea']

and another file docker-compose-dev.yml with the following contents (replace "${my-project}" with your project name):

version: '3'
services:
  tomcat:
    volumes:
      - ${my-project}-sync:$PWD:nocopy

volumes:
  ${my-project}-sync:
    external: true

Now instead of using docker-compose up or docker-compose start to start Docker, use docker-sync-stack start. Docker Sync automatically detects docker-compose.yml and docker-compose-dev.yml and ignores docker-compose.override.yml, so either style can be used depending on how you start Docker. Note that this starts a second Docker container, "${my-project}-sync".

Errors Installing Docker Sync

Some Mac users have reported the error "ERROR: mkmf.rb can't find header files for ruby at /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/include/ruby.h" when running docker-sync-stack start for the first time. If you encounter this, you may need to install the xcode command line tools:

xcode-select --install
sudo xcodebuild -license
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Useful commands and tips

Add this function to your ~/.bash_profile:

function d() {
  docker-compose exec $1 bash --login;
}

Use this to log in to each container by name from the project directory, for example:

$ d tomcat
[brightspot tomcat:~]$ cat /servers/tomcat/conf/context.xml

These commands can be executed on the command line if your current working directory is inside the project.

• Restart a single container
  • docker-compose restart tomcat
• Stop the project docker container (This does not delete data)
  • docker-compose stop
• Start a stopped docker container
  • docker-compose start or docker-compose start -d to detach and run it in the background
• Delete container
  • docker-compose down
  • Note: This does not delete data stored in named volumes.
• Delete unused volumes
  • docker volume prune
• Start with Docker Sync
  • docker-sync-stack start
• Tail the tomcat logs
  • docker-compose logs -f tomcat
• Increase the memory available to Docker to 4 GB
  • https://forums.docker.com/t/how-to-increase-memory-size-that-is-available-for-a-docker-container/78483⁠
• Run an interactive shell in the project Docker container
  • docker-compose exec tomcat bash
• Restore database backups
  • First shut down all containers:
    • docker-compose stop or Ctrl+C if running in the foreground
    • NOTE: This will only work if the mysql service has been started successfully at least once.
  • Then run the restore script on the mysql service:
    • docker-compose run mysql restore mysql -e qa --user ${UserName} --project ${DatabaseName} --account ${Account} --renamedb ${Project} --region=${Region}
      • Container - container name typically project name
      • UserName - Ldap Username
      • Project - Project Name
      • DatabaseName - Typically project name
      • Account - AWS account
      • Region - (Optional) AWS Region
    • Example docker-compose run mysql restore mysql -e qa --user bob --project myproject --account psd-myproject --renamedb myproject --region=us-west-1
  • Then restart all containers:
    • docker-compose start