Generate Docker Containerized Run Time and Design Time for Oracle StreamExplorer, Event Processor and JDeveloper using Vagrant, Puppet and VirtualBox image152

Generate Docker Containerized Run Time and Design Time for Oracle StreamExplorer, Event Processor and JDeveloper using Vagrant, Puppet and VirtualBox

The objective of this article: prepare an environment for experimenting with Oracle StreamExplorer using two Docker containers inside a Linux VM running on my Windows laptop. And with the smallest number of manual steps possible – as to make the environment provisioning process highly repeatable.

In recent weeks, I have published a number of articles, investigating pieces of the puzzle that come together in today’s challenge. After general introductions of the combination of Vagrant and Docker – First steps with provisioning of Docker containers using Vagrant as provider and Vagrant and Docker – Next and Advanced steps with folders, ports, volumes, linking and more – that explained how Vagrant can be used on a Windows host to easily build, run and manage Docker containers – I discussed a number of more advanced topics such as the use of Puppet to do complex environment provisioning for the Docker containers – Vagrant and Docker followed by Puppet to provision complex environments – and how to configure and containers with GUI applications – Vagrant, Docker, VirtualBox and the Graphical Desktop for GUI applications in Docker Containers.

I have also discussed at length how to install OEP – using Puppet and without using Puppet: Generation of VM image for Oracle Event Processor and Stream Explorer using Vagrant and Puppet and Oracle StreamExplorer and Oracle Event Processor – installation instructions to quickly get going.

.

With all these findings under my belt, I felt ready to take on the challenge.

This article describes what I did and what the final result was – as visualized in this figure:

image

The sources – all Vagrant, Docker and Puppet configuration files – can be found on GitHub: https://github.com/lucasjellema/vagrant-docker-puppet-desktop-streamexplorer-jdeveloper.

The preparation for the steps described next consists of:

  • Installing Vagrant and VirtualBox on the host
  • Download installation files for Java (JDK 7u79), Oracle Event Processor and StreamExplorer and for JDeveloper (the one that is part of the SOA Suite Quick Install); these files are moved to the /files directory in the project cloned from GitHub

The directory structure after cloning the project:

image

The files directory contains these files:

image

The steps to produce the environment with the DockerDesktopHostVM with two Docker Containers with StreamExplorer/OEP and JDeveloper respectively are these:

1. Use Vagrant to create two base containers (my-base-container and my-desktop-base-container) in doing so also create the VirtualBox DockerDesktopHostVM virtual machine with Ubuntu 14.04 plus Lubuntu desktop. This is onde by running two statements from the root of the project directory:

vagrant up my-desktop-base-container

and

vagrant up my-base-container

The first statement makes Vagrant prepare the container we will use for running JDeveloper:

image

Step 9 in the Dockerfile is different from the my-base-container: in this step we install the Xterm libraries that allow for exposing the GUI.

image

The second statement produces the my-base-container, a headless Ubuntu 14.04 container inside the dockerdesktophostvm.

image

the last section of this vagrant operation shows some interesting details: port 9002 is exposed (that is the port OEP will be running on) and two directories on the host are mapped into the container and will be available as /stage and /puppet when we start provisioning the container using Puppet.

image

 

2. Connect into both containers and use Puppet to do the detailed provisioning and configuration

With vagrant global-status we learn the identifier for the dockerdesktophostvm and using vagrant ssh and that identifier, we can open a terminal session into the dockerdesktophostvm. With docker ps -a we find the container identifiers for the two containers we had Vagrant build earlier on.

SNAGHTML1df34ee

Using docker exec -it <container identifier> bash we open a session in the container.

To activate Puppet to provision the container, we use:

puppet apply –debug –modulepath=/puppet/modules /puppet/manifests/base.pp

in my-base-container and

puppet apply –debug –modulepath=/puppet/modules /puppet/manifests/jdev.pp

in my-desktop-base-container.

image

 

The Puppet modules and manifests are available inside the container thanks to the mapping in the Vagrant file.

Here is what happens when we run Puppet in my-base-container:

image

Java (JDK 7u79) is installed. Subsequently, OEP is installed and the Patch for StreamExplorer is applied. See modules oep and opatch and the main manifest file base.pp for details.

The final step by Puppet is the ‘creation’ of the OEP domain by copying a set of files from the files/user_projects directory on the host (part of the project on GitHub) and inspired by Guido Schmutz (https://guidoschmutz.wordpress.com/2015/03/29/installing-oracle-stream-explorer-in-a-docker-image/).

image

At this point, we can exit the container.

 

3.Create Docker Images from the Docker Container

Then we commit the container and create an image out of it:

docker commit containerId  me/sx_12_1_3:1.0

image

 

Now we have an image that can be used to run containers with.

And for the my-desktop-base-container:

image

4. Running containers from the Docker Images

The Simple Docker UI (that I discussed earlier in this article: Simple Docker GUI for monitoring and managing containers and images – in combination with Vagrant and VirtualBox) provides a neat overview of our images:

image

We can now run a container based on one of these images. When we do so for image me/sx_12_1_3:1.0, we need to expose the port OEP is running on (9002) and ensure that the OEP server started for the sx_domain. The statement used in the dockerdesktophostvm to run the SX/OEP container is:

docker run -p 9202:9002 –user oracle -w /u01/app/oracle/OEP_Home/user_projects/domains/sx_domain/defaultserver –name OracleStreamExplorer  me/sx_12_1_3:1.0 /bin/bash “startwlevs.sh”

The -p switch takes care of port mapping between dockerdesktophostvm (9202) and the Docker container (9002). OEP/SX can be accessed in the host at port 9202. The -w parameter specifies the working directory where the command to be executed will be executed. The -u switch indicates the Linux user as which the command should be executed. With –name we can set a name for the container that makes it easier to recognize and to link to or use volumes from. Finally the start-up script for OEP is startwlevs.sh

image

After running the container with this command, we can access StreamExplorer from the Vagrant host at http://10.10.10.30:9202/sx. Note that 10.10.10.30 is the IP address for the dockerdesktophostvm on the private network defined in the DockerHostVagrantFile.

image

 

In order to work with JDeveloper – a GUI application – we need to run the container from within the DockerDesktopHostVM that is capable of sharing its X-Windowing with the container. The command we need to use for running the container is somewhat complex. It has to forward the display system, link to the OracleStreamExplorer-container that was started earlier on, run the JDevelopet startup script etc.

docker run -v /vagrant/files:/my-stage  -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY –link OracleStreamExplorer:sx  –name JDeveloper_12_1_3 –user oracle -w /u01/app/oracle/JDEV_Home/jdeveloper/jdev/bin  me/jdeveloper_12_1_3:1.0 “./jdev”

Note the –link parameter that makes the container OracleStreamExplorer available and accessible inside the container using the alias sx.

This figure shows how the JDeveloper GUI is launched inside the dockerdesktophostvm when the Docker container is run using this statement:

image

In JDeveloper we need a connection to the OEP/SX server running in the  OracleStreamExplorer  container – to stop and start the server, deploy applications etc. This connection is created in the regular way:

image

The name of the host is sx – thanks to the link configuration between the Docker containers. Note that we access the container on the regular port – ignoring the port forwarding to the host (9002 => 9202)

Once the connection is created, we can inspect the remote OEP server:

image

 

The Simple Docker UI shows two containers running

image

and can provide more useful details if we like

image

 

Summary

The picture says it all.

image

We start with a host that contains nothing but Vagrant and VirtualBox and a bunch of virgin installation files. We run a few scripts, wait for 20-30 minutes and we have a fairly complex environment for doing StreamExplorer development. And with hardly any effort, we can repeat the process and recreate this environment. An environment that is so virtualized and isolated that we can throw it away at any time without impacting our host machine.

 

Resources

The sources on GitHub:

https://github.com/lucasjellema/vagrant-docker-puppet-desktop-streamexplorer-jdeveloper.

My earlier articles:

General introductions of the combination of Vagrant and Docker – First steps with provisioning of Docker containers using Vagrant as provider and Vagrant and Docker – Next and Advanced steps with folders, ports, volumes, linking and more 

More advanced topics such as the use of Puppet to do complex environment provisioning for the Docker containers – Vagrant and Docker followed by Puppet to provision complex environments – and how to configure and containers with GUI applications – Vagrant, Docker, VirtualBox and the Graphical Desktop for GUI applications in Docker Containers.

How to install OEP – using Puppet and without using Puppet: Generation of VM image for Oracle Event Processor and Stream Explorer using Vagrant and Puppet and Oracle StreamExplorer and Oracle Event Processor – installation instructions to quickly get going. On Vagrant, VirtualBox and Puppet with Ubuntu 14.04 on Windows Quickly produce a Linux 64 bit Ubuntu 14.04 Desktop environment using Vagrant and Puppet – as starting point for Oracle installations – (Ubuntu 14.04 64bit plus Desktop and JDK 7U79)

Introduction to the Simple Docker UI: : Simple Docker GUI for monitoring and managing containers and images – in combination with Vagrant and VirtualBox)

Guido Schmutz: Providing Oracle Stream Explorer environment using Docker uses a different approach: no Puppet is used, no Vagrant is explicitly visible. His article provided important elements and inspiration to mine.

 

Download Oracle Software: http://www.oracle.com/technetwork/middleware/complex-event-processing/downloads/index.html.

Downloads for Java (JDK): http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html