Next step with Docker – Create Image for Oracle Database 12c using Puppet

8
Share this on .. Tweet about this on TwitterShare on LinkedIn53Share on Facebook0Share on Google+0Email this to someoneShare on Tumblr0Buffer this page

In a previous article – https://technology.amis.nl/2015/03/15/docker-take-two-starting-from-windows-with-linux-vm-as-docker-host/ – I have discussed my first steps with Docker. With Windows as my host environment I used Vagrant to create a VirtualBox VM with Ubuntu. In that VM I installed Docker can played around creating some containers, images and eventually an image for Oracle WebLogic 12.1.3. I leveraged the excellent work by Mark Nelson (especially his article https://redstack.wordpress.com/2014/11/14/gettingn-to-know-docker-a-better-way-to-do-virtualization/).

In this article I am taking things one step further by creating a Docker container – and from that container an image – with the latest Oracle Database release 12.1.0.2 (Enterprise Edition). Again, the Mark Nelson article is my guide and Edwin Biemond – champion of all things automated – provided the Docker file and Puppet scripts that get the job done. Edwin was also kind enough to help me out when a library dependency caused problems.

I ran into the (default) size limitation on Docker containers (10 GB) while installing the Oracle Database. I resolved this challenge by mapping a host folder to the container (with the original database software) and by sharing a volume from a second container that was used as temporary (staging) area. Thus I virtually expanded the file system of my container considerably beyond the 10 GB mark.

The steps I went through are basically:

0. preparation: (as discussed in the previous article) Get a Ubuntu based Virtual Machine (Virtual Box) running on my Windows host laptop. Install Docker into this VM. Download the Oracle Database software (12.1.0.2 Enterprise Edition) from http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index-092322.html. Two files are downloaded to a folder on the host with a total of some 2,6 GB.

image

Note: the Puppet scripts expect two files with names linuxamd64_12c_database_1of2.zip and linuxamd64_12c_database_2of2.zip. I renamed the downloaded files to match these expectations.

1. create Docker image – db12c102_centos6 –  based on CentOS with Puppet installed as well as the Puppet modules required for the database installation

2. create Docker container – softwarecontainer – to act as the ‘staging container’ – a container that shares a volume that can be used as expanded file storage from other containers

3. run Docker container based on image db12c102_centos6 with host folder mapped into it and with volume shared from softwarecontainer

4. edit Puppet files to match the 12.1.0.2 Enterprise Edition installation

5. run the Puppet site.pp manifest – this will install the database software and initialize an instance

6. test whether the container is actually running the database; then create an image from the container

 

At the end of the article, I have both a container and image with the Oracle Database 12c (12.1.0.2) running – based on the oradb Puppet Module by Edwin Biemond. The container exposes port 1521 where the database can be accessed from the host as well as from other containers – as we will see in subsequent articles.

1. Create base Docker image

With a simple Dockerfile – derived from Docker Database Puppet – I create a base image that provides the starting point for the container that will hold the Oracle Database installation.

The Dockerfile looks like this:

# CentOS 6
FROM centos:centos6

RUN yum -y install hostname.x86_64 rubygems ruby-devel gcc git unzip
RUN echo “gem: –no-ri –no-rdoc” > ~/.gemrc

RUN rpm –import https://yum.puppetlabs.com/RPM-GPG-KEY-puppetlabs && \
rpm -ivh
http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

# configure & install puppet
RUN yum install -y puppet tar
RUN gem install highline -v 1.6.21
RUN gem install librarian-puppet -v 1.0.3

RUN yum clean all

ADD puppet/Puppetfile /etc/puppet/
ADD puppet/manifests/site.pp /etc/puppet/

WORKDIR /etc/puppet/
RUN librarian-puppet install

EXPOSE 1521

ADD startup.sh /
RUN chmod 0755 /startup.sh

WORKDIR /

CMD bash -C ‘/startup.sh’;’bash’

The image is created from this Dockerfile using this command:

docker build -t db12c102_centos6 .

A screenshot from somewhere midway in the creation process:

image

and the completion:

image

Note: Initially I used the exact same Dockerfile Edwin published – with an edited site.pp file in order to install the 12.1.0.2 Enterprise Edition instead of the 12.1.0.1 Standard Edition. I then ran into disk space limitations. Apparently, copying the software zip files (2,6 GB) to the container and extracting the content of these files inside the container occupied so much space that the installation was aborted.

image

I needed a trick to create container and install the Oracle Database without burdening the container with these zip-files and their temporary extracted contents. The trick consists of three things:

  1. create the image in multiple steps (instead of a single one with a single Dockerfile that auto-runs the complete Puppet script), starting with a simple base image
  2. run a container from this base image with a host folder mapping from which it can access the software zip-files without actually copying them to the container’s file system
  3. have the container import a volume from another container; this volume is used as temporary staging area (for the extracted files needed only during installation); finally create an image from this last container

Also note that in the original script, Edwin did not have the line “RUN gem install highline -v 1.6.21”; he advised me to add this line because the original Dockerfile resulted in a dependency error:

image

Adding this line (to make sure highline gets installed before a version of highline with a more demanding requirement on Ruby is brought along by librarian-puppet.

2. Create Docker container – softwarecontainer – to act as the ‘staging container’

A very simple Docker image is created using this Dockerfile:

FROM busybox

RUN mkdir /var/tmp/install

RUN chmod 777 /var/tmp/install

VOLUME /var/tmp/install

VOLUME /stage

CMD /bin/sh

and this command:

docker build -t softwarestore .

This results in an image called softwarestore which exposes its folder /var/tmp/install as a volume that can be used as expanded file storage from other containers.

Start the container softwarecontainer based on image:

docker run -i -t -v /host_temp/shared:/stage/shared –name softwarecontainer softwarestore /bin/sh

The container softwarecontainer is now available along with its /var/tmp/install volume that will be used during database installation as staging area.

 

3. Run Docker container based on base image db12c102_centos6

Run a new container based on the base image created in step 1:

docker run -ti  -v /host_temp/shared/downloads:/software –volumes-from softwarecontainer  db12c102_centos6  /bin/bash

with host folder mapped into it and with volume shared from softwarecontainer.

The host folder with the database software is accessible from within the container at /software, as is the /var/tmp volume in the softwarecontainer:

image

 

 

4. Edit Puppet files to match the 12.1.0.2 Enterprise Edition installation

Inside the container: Open the site.pp file at /etc/puppet in a text editor. Note: this directory and this file were created along with the base image in step 1.

image

Edit the lines that refer to SE (Standard Edition) and 12.1.0.1:

image

Note that only a few changes are required to process EE instead of SE and 12.1.0.2 instead of some other version.

 

5. Run the Puppet site.pp manifest to install the database software and initialize an instance

The heavy lifting regarding the installation of the Oracle Database and the creation of an instance (service orcl) is done by Puppet. The Puppet script is started (still inside the container) using this command:

puppet apply /etc/puppet/site.pp –verbose –detailed-exitcodes || [ $? -eq 2 ]

The first steps are shown here:

image

And the last ones:

image

When Puppet is done, we have a running database. All temporary files have been cleaned up.

6. Test whether the container is actually running the database – then create an image from the container

With these commands (inside the container) we can run SQL*Plus and connect to the running database instance:

export ORACLE_SID=orcl

export ORACLE_HOME=/oracle/product/12.1/db

cd $ORACLE_HOME/bin

./sqlplus “sys/Welcome01 as sysdba”

SQL*Plus is started and we can for example select from dual.

image

Note: The database sid = orcl. Password for SYS and SYSTEM are Welcome01.

Using exit twice – once to leave SQL*Plus and once to exit the container, we return to the host. The container is shown (agitated_bohr).

The next step – which takes some time, due to the size of the container and the images created from it – is to create an image that captures the current state of the container:

docker ps – a (to find the container id)

docker commit <container id>

docker images

assign nice name and version to image:

docker tag c5d3effcbdd6 oracle-db-12-1-0-2:1.0

Look at the result of this:

image

A sizable image – that through export and import and be reduced in size although that would severe the link with the base centos image.

The situation at this point can be visualized using this picture:

image

 

7. Run Container based on the Image

Run container from that image, local folder /software mapped to host folder that contains the software

docker run -ti -p 0.0.0.0:1521:1521 -v /host_temp/shared/downloads:/software --volumes-from softwarecontainer oracle-db-12-1-0-2:1.0  /bin/bash

Note: the -v and –volumes-from are not really required, because the two folders were required only for installing the database (during the creation of the image). Running the container with:

docker run --privileged=true -ti -p 1521:1521 oracle-db-12-1-0-2:1.0 /bin/bash

will do the job just as well. Note: I have added the –privileges=true here because I ran into a problem with not being able to switch users in the container. This discussion led me to use this additional parameter.

Once the container is fired up, the database can be started – using the /startup.sh script or using the statements listed in step 6. That is: I went through these steps (which seem a little bit more than should be required):

su oracle

/startup.sh

[provide oracle as the password]; this will start the listener

export ORACLE_SID=orcl

export ORACLE_HOME=/oracle/product/12.1/db

cd $ORACLE_HOME/bin

./sqlplus “sys/Welcome01 as sysdba”

SQLPlus starts and connects us to an idle instance. Then type startup – and the database is started.

After exiting SQL*Plus, I can check the listener status:

./lsnrctl

then type status.

The service orcl.example.com and the instance orcl are both ready.

Tidbits

These are really notes to myself – little things I needed or came across while going through the steps described in this article.

Docker

Docker stores container data in the directory /var/lib/docker/containers in Ubuntu.

Remove a single image: docker rmi IMAGE_ID (note: images can only be removed when no containers are based off of it)

Trick for removing multiple images in one go: http://stackoverflow.com/questions/17665283/how-does-one-remove-an-image-in-docker

Remove Docker Containers (http://stackoverflow.com/questions/17236796/how-to-remove-old-docker-containers):

docker rm $(docker ps --before <Container ID> -q)

Volumes are outside of the Union File System by definition, so any data in them will not count towards the devicemapper 10GB limit (http://fqa.io/questions/29083406/docker-disk-quotas). By adding a VOLUME in the Dockerfile, I am hoping to be able to leave the big software install packages outside the 10GB limit. image Learn the IP address assigned to a container:

$ sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' <container_id>

When a Docker container is restarted its, IP Addresses changes. Applications as well as others servers that were communicating with the container before the container restart, will be unable to communicate. Configuring a DNS server on Docker and configure consumers to use DNS names is a solution to the IPAddress change after a container restart.

Interesting observations in this white paper from Oracle on WebLogic and Docker

Linux

Determine size on disk of directories and their contents: du -f  (http://www.codecoffee.com/tipsforlinux/articles/22.html)

Remove entire directory: rm -r directory

Library Dependency

While creating the Docker image using Edwin’s Docker file, I ran into a dependency issue, that Edwin helped me fix. (well, he fixed it for me)

As the screenshot shows, the highline component that apparently gets installed as part of librarian-puppet requires a higher Ruby version than is available.

image

This was resolved by adding a single line to the Docker file:

RUN gem install highline -v 1.6.21

just prior to the line that installs librarian-puppet. This makes sure that highline – a version that does not have this advanced Ruby dependency – is already around when the librarian-puppet is installed. It will therefore not try to install the latest version of highline, that gives us the problem.

Share this on .. Tweet about this on TwitterShare on LinkedIn53Share on Facebook0Share on Google+0Email this to someoneShare on Tumblr0Buffer this page

About Author

Lucas Jellema, active in IT (and with Oracle) since 1994. Oracle ACE Director and Oracle Developer Champion. Solution architect and developer on diverse areas including SQL, JavaScript, Docker, Machine Learning, Java, SOA and microservices, events in various shapes and forms and many other things. Author of the Oracle Press books: Oracle SOA Suite 11g Handbook and Oracle SOA Suite 12c Handbook. Frequent presenter on community events and conferences such as JavaOne, Oracle Code and Oracle OpenWorld.

8 Comments

  1. Hi Lucas,
    Thanks a lot for your article.
    The steps where it says open /etc/puppet site.pp
    i see the site.pp located for me under manifests. not sure is this is the right one. I donot have one under /etc/puppet.

    Also, i do not have much info in the site.pp
    My site.pp has only the following info.
    Exec { path => “/usr/local/pdit/bin:/usr/local/git/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin/:/bin:/sbin” }

    import “classes/*.pp”
    #import “groups/*.pp”
    import “nodes/*.pp”

    where as the steps shows all the oracledb12c info is inside it.

    Could you please let me know what i was missing?

    thanks in advance.

  2. Hi i tried your guide but it will stops at step 11 (RUN librarian-puppet install) telling that

    /bin/sh: librarian-puppet: command not found

    i entered into the container and see that librarian-puppet is installed (setp 6) but i cannot run it, i can’t find the executable. can you help me? thanks

    • ok, i resolved, pay attenction at what you paste on the shell, because some chars must be rewritten, for example the ‘double minus’ chear into Dockerfile

  3. Hi Lucas

    Thanks a lot for your article.
    When i build the docker image base ( step 1) , i obtain this error :

    Step 11 : ADD puppet/Puppetfile/ /etc/puppet/
    puppet/Puppetfile/: no such file or directory

    Can i do it to fix this problem ?

    Thanks

    • Hi Agasse,

      Did you get the file puppet/Puppetfile, am also facing same issue. Please let me know.

      Thanks

Leave a Reply