Public | Automated Build

Last pushed: 4 months ago
Short Description
Speed up the creation of a MarkLogic docker container by pre-installing dependencies
Full Description

This speeds up the creation of a fresh MarkLogic docker instance by including
all necessary dependencies. You will have to separately download a MarkLogic Early Access
rpm for CentOS 7
and move the rpm to
the same folder as your Dockerfile.

To use, you will create a local image with MarkLogic installed - an image that
can then be used over and over to create fresh MarkLogic containers. (Note that
an image with MarkLogic pre-installed cannot be shared. It has the same
licensing restrictions that a MarkLogic rpm has.)

To create this local image with MarkLogic installed, first create a Dockerfile like this (probably in a new directory):

FROM patrickmcelwee/marklogic-dependencies:9.0-EA4

# Install MarkLogic
WORKDIR /tmp
ADD MarkLogic-9.0-20170113.x86_64-EA4.rpm /tmp/MarkLogic.rpm

RUN yum -y install /tmp/MarkLogic.rpm && yum clean all && rm /tmp/MarkLogic.rpm

# Expose MarkLogic Server ports - plus 8040, 8041, 8042 for your REST, etc
# endpoints - feel free to add more
EXPOSE 7997 7998 7999 8000 8001 8002 8040 8041 8042

# Define default command (which avoids immediate shutdown)
CMD /etc/init.d/MarkLogic start && tail -f /dev/null

Then add your MarkLogic rpm to the
same directory. This command will build an image with MarkLogic installed, but
not initiated:

docker build -t marklogic-initial-install:9.0-EA4 .

That will download this image automatically and build a new image called
marklogic-initial-install:9.0-EA4. You can then create and run a new container
based on this image with the following command:

docker run -d --name=initial_install -p 8001:8001 marklogic-initial-install:9.0-EA4

Now navigate to http://localhost:8001. This will cause MarkLogic to complete
installation and allow you to set up cluster settings and an admin user. I like
to create an image immediately after taking these steps, so that I can create
fresh images without any manual setup:

docker commit initial_install ml-installed:9.0-EA4

That creates a local image named ml-installed:9.0-EA4 that is ready to go
with the cluster and admin user you set up! (Note that docker commit can be
used to save the state of the container, even once you start adding data. This
can be handy to create a base state to which you can easily rollback. I am not
sure if that is possible when using external volumes, which I do not cover
here. Feedback welcome on this point!)

Now you can cleanup that intermediate initial_install container:

docker stop initial_install
docker rm -v initial_install

Now you can quickly create new containers based on your installed-MarkLogic
image, and you will have completely isolated MarkLogic servers.

docker run -d --name=name_for_this_container -p 7997:7997 -p 7998:7998 -p 7999:7999 -p 8000:8000 -p 8001:8001 -p 8002:8002 -p 8040:8040 -p 8041:8041 -p 8042:8042 ml-installed:9.0-EA4

When you are finished with this container, you can remove it, but remember the
-v option, if you have set up a separate volume. Otherwise, the volume
will not be removed:

docker rm -v name_for_this_container

Happy Docking!

Docker Pull Command
Owner
patrickmcelwee

Comments (9)
patrickmcelwee
4 months ago

I still don't get email notifications of comments, despite my settings looking right :-/

patrickmcelwee
4 months ago

@tomersha Easy to answer. I just didn't think of it. :-) Seems like a good addition.

I'm curious: Are you using this image from outside MarkLogic? I'd like to gauge if others are using this. MarkLogic recently stood up an internal registry for images with MarkLogic pre-installed (which we can't distribute for licensing reasons). I would probably use the internal images myself, unless others are making use of this image.

tomersha
4 months ago

Hey Patrick,

I wonder why you are not issuing yum clean all after yum install (line 4 in the Dockerfile).

This is also explained here: http://developer.marklogic.com/blog/building-a-marklogic-docker-container

The yum clean all command cleans up any yum cache to help keep our image a reasonable size

patrickmcelwee
8 months ago

@millerbill3 Thanks! I removed that system variable from the MarkLogic 9 version of this. I could backport that if that's helpful.

patrickmcelwee
8 months ago

@mseddon,

What I do is to create an initial container, go into the UI and manually set it up, then use docker commit to make a new image that contains the results of those manual actions. That is the image that I actually use to run new containers. (See the full description above.)

mseddon
9 months ago

Hey Patrick,

I've got something semi-working using your code as a base - thanks!

What I'm looking for is an elegant solution to automatically configuring the ML instance so that it is 'ready to go' on container startup.

E.g I've been trying to run 'curl -X POST -d "" http://127.0.0.1:8001/admin/v1/init' (and related curls) to bypass the need to go via the UI on startup. The issue I'm having is that it's obviously not possible to hit this endpoint as it does not exist until the container starts up.

If I start the ML container & then execute my required commands (some curls and a maven goal to pre-populate some data) it all works great. The manual intervention is not so great.

Any thoughts on this?

millerbill3
10 months ago

FYI... I discovered that the original Dockerfile used to build this image includes LD_LIBRARY_PATH environment variable. This will cause certain library links to break. In my instance, Yum and Curl broke. Removing this environment variable and rebuilding the image works.

patrickmcelwee
a year ago

@pablodelpino ... yes, you can use Roxy (./ml local bootstrap, etc). To do that, you have to set local-server={docker ip} in deploy/local.properties (or, less preferably, directly in deploy/build.properties

Sorry for not answering earlier. I wasn't notified about your comment. If you'd like to talk more, email me at patrick.mcelwee@marklogic.com

pablodelpino
a year ago

HI Patrick, thanks for this, I've been running a modified version of your docker with CentOS 7. I was wondering if this supports deployments via ml local bootstrap?

I'd love to pick your brain about it, if it's possible.