SOA Suite Build, Deployment and Test Automation – part 1

In the latest SOA Suite release Oracle has improved the automated build capabilities. Instead of the Oracle BPEL specific Ant tool (obant, a customized version of Ant) plain vanilla Ant is now used for building and deploying BPEL suitcases. Let’s put it to the test: subject of this blog post is the set up of an automated build and deployment environment for SOA Suite. The goal is to automatically build Oracle’s SOA Order Booking demo application and deploy it to an Oracle Application Server that runs the SOA Suite. Oracle’s demo application is selected because it uses all components in the SOA Suite: the brand new ESB, BPEL and Business Rules. The demo application also includes a number of Web Services that are implemented in Java.

This is the first in a small series of posts. Focus of part 1 is on describing the nuts and bolts of the automated build environment and subsequently on building the Web Services and BPEL components in the SOA demo application. Goal of part 2 is to automatically build and deploy the ESB components. Finally, in part 3, I will focus on the new unit test capabilities of Oracle BPEL processes.

Rationale

The last two years I have been working with Oracle SOA products, especially BPEL. In my current project a team of approximately 8 people develops and maintains a fairly large Oracle BPEL based system. One of our responsibilities is to keep the development, test and production environments up to date. Also, we regularly set up new environments, e.g. for testing implementations for new clients of the customer. Especially when we work towards a new release the team rolls out new releases on a daily basis. Although we use Subversion and the build procedure is supported with Ant, one person in the team constantly finds himself occupied with release management and troubleshooting. There is a need for improved control and further automation of this process. Therefore, the team is investigating the use of Luntbuild, a build automation and management tool. Also, the customer plans a migration from the current BPEL release 10.1.2 towards the latest Oracle SOA Suite release 10.1.3.1.

Clear business value

A Service Oriented Architecture (SOA) bridges the gap between business and information technology. In a typical SOA environment, business processes are constructed from a variety of business services. Each business service may be implemented using different information systems or technologies making a SOA implementation a true integration effort. But the effort is worth your while as all the architectural benefits of the evolution in enterprise integration made their way into SOA. To name a few:

• Widely adopted Web Services standards allow easy assimilation of specialised components (favouring buy over build) and
• Loosely coupled services that are based on proven messaging technologies provide a highly scalable and robust platform.

In the end, all these loosely coupled pieces of technology, each having their own versions and release schedule, need to interoperate for the system to work. That makes release management and integration testing in a SOA environment an important and potentially daunting task. Implement strict procedures and appropriate tools to control this from the very beginning. As this blog post outlines it is not that hard to automate these repetitive and tedious build and deployment tasks. In the end this will save you valuable time and helps to provide your customers with consistent quality systems.

Nuts and bolts for creating your own build and deployment server

For my build server I installed a number of components. Obviously, to start with I installed an Oracle XE database and Oracle Application Server 10.1.3.1 with the accompanying Oracle SOA Suite. No need to install the Ant build tool as it is bundled with the Application Server.

Luntbuild

The open source tool Luntbuild is not only selected because it is used in my current project. The main reason is that Luntbuild is able to deal with dependencies between different projects. In Luntbuild, a project is the metaphor for anything that you want to build, a ‘buildable unit’ in Luntbuild terms. For example, it is possible to build and deploy a Web Service first and when that task is done, build and deploy the BPEL process that uses the Web Service. Also, Luntbuild (optionally) stores its build data in an Oracle database allowing you to easily track, manage and report on your daily builds.

Subversion

My source code version control system of choice is Subversion. But Luntbuild does not discriminate; it supports a wide variety of version control systems like CVS, Visual Sourcesafe or Clearcase. Subversion features a powerful command-line interface. Also, there is a great plug-in for Oracle JDeveloper. Alternatively, you can use graphical clients like Tortoise or SmartSVN to access a Subversion repository. I created my repository and subsequently retrieved a working copy using the following two lines:

svn import C:\oracle\product\soademo file:///c:/projecten/svnrepos/soademo -m "Initial import"
svn checkout svn://localhost/projecten/svnrepos/soademo soademo

Notice the uri in the checkout command that starts with ‘svn://’. Subversion also supports the file based interface, the similar command being:

svn checkout file:///c:/projecten/svnrepos/soademo soademo

The samples provided for the Luntbuild configuration indicate that this file-based interface can be used. Using it, I stumbled upon a known problem. Hence, we need a server for accessing Subversion. In larger, multi-user environments, Subversion is probably accessed via an Apache server. For my single-user build system Svnserve provides a lightweight and very easy to use stand-alone alternative. Svnserve is already bundled with Subversion.

Ant build scripts for the Web Services components

The Quick Start guide for the SOA Order Booking application assumes that all components are installed from JDeveloper. In order to automate builds, we need build scripts. For Oracle BPEL projects these Ant build scripts are created automatically by JDeveloper. The demo application does not come with build scripts for the Web Services neither for the ESB components. In this paragraph I outline the creation of an Ant build script for the CreditService.

A good starting point is the Ant build script generator in JDeveloper that can be executed to create an Ant build file for the project as it is defined in JDeveloper. Use it to generate a skeleton that at least contains the classpath settings for the Java libraries in the project.

Than, add Ant targets for compiling the Java classes that comprise the Web Service as well as for creating Web and Enterprise archive files. The latter two are shown here:

<!-- Create war file -->
<target name="warfile" depends="rebuild">
  <echo message="Creating the Web Application module (war) file"/>
  <delete>
		<fileset dir="${dir.deploy}" includes="${warfile.name}"/>
	</delete>

  <war destfile="${dir.deploy}/${warfile.name}"        
       webxml="${dir.html.web-inf}/web.xml">
    <classes dir="${dir.output}" includes="**/*"/>
    <webinf dir="${dir.html.web-inf}" includes="**/*"/>
  </war>    

  <!-- Clean up the build information -->
  <delete dir="${dir.output}"/> 
</targ
et>

<!-- Creat
e ear file -->
<target name="earfile" depends="warfile">
  <echo message="Creating Enterprise Archive (ear) file"/>
  <delete>
	<fileset dir="${dir.deploy}" includes="${appl.name}.ear"/>
  </delete>

  <ear destfile="${dir.deploy}/${appl.name}.ear" appxml="${dir.deploy}/application.xml">
    <fileset dir="${dir.deploy}" includes="*.war,orion-application.xml"/>
  </ear>
</target>

Finally, use the oracle:deploy Ant task for OC4J that comes with the Oracle Application Server. This task utilizes the OC4J client administration tools in order to deploy a module to OC4J. Oracle provides clear instructions for use with OC4J standalone, for the container running in an Application Server setting as well as for a clustered environment. Simply add a namespace declaration to your Ant build file, extend the build path so that Ant can access the Oracle Ant task specific libraries and add the oracle:deploy target to the build.xml file:

<target name="deploy" depends="earfile">
  <echo message="Deploying Enterprise Archive (ear) file"/>
    <oracle:deploy deployerUri="${deployer.uri}"
                   userId="${oc4j.admin.user}"
                   password="${oc4j.admin.password}"
                   file="${dir.deploy}/${appl.name}.ear"
                   deploymentName="${appl.name}"
                   bindAllWebApps="default-web-site"
                   logFile="${dir.deploy}/deploy-${appl.name}-ear.log"/>
</target>

The important thing here is to have the deployer URI right.
Whether the build was successful can be tested with the Application Server Control that comes with a facility to test Web Services. This is shown in the following screenshot:

Create similar build scripts or enhance this Ant script for building and deploying the CustomerService and the RapidService of the SOA Demo Application. Take into account that these use different libraries.

Building BPEL Suitcases using Ant

In SOA Suite 10.1.3.1 standard Ant scripts are used for building and deploying BPEL projects (suitcases). The build.xml file for a new BPEL project is generated automatically when a new BPEL project is created in JDeveloper. It is great to see that the Ant build scripts are now actually used by JDeveloper for BPEL deployments.

When running Ant for building the SOAOrderBooking BPEL process I got the message BUILD FAILED: Error while deploying decision services. Setting verbose=”true” on the deployDecisionServices task reveals the problem:

deployDecisionServices:
     [echo]
     [echo] --------------------------------------------------------------
     [echo] | Deploying decision services for SOAOrderBooking on localhost, port 80
     [echo] --------------------------------------------------------------
     [echo]
[deployDecisionServices] Start of deploying decision services.
[deployDecisionServices] Deploy decision service in directory C:\projecten\soademo\SOAOrderBooking\decisionservices\.svn
[deployDecisionServices] Start deploying decision service from directory C:\projecten\soademo\SOAOrderBooking\decisionservices\.svn to J2EE context /rules/default/SOAOrderBooking/1.0/.svn
[deployDecisionServices] Replace placeholders in file C:\projecten\soademo\SOAOrderBooking\decisionservices\.svn\war\WEB-INF\wsdl\.svn.wsdl
[deployDecisionServices] Replace placeholders failed for C:\projecten\soademo\SOAOrderBooking\decisionservices\.svn\war\WEB-INF\wsdl\.svn.wsdl
[deployDecisionServices] Error in ant execution: Replace placeholders in WSDL file failed

BUILD FAILED
C:\projecten\soademo\SOAOrderBooking\build.xml:126: Error while deploying decision services on server "localhost"
 

It turns out that the .svn directory that is used by Subversion for administration purposes clutters things up. Apparently, Oracle’s deployDecisionServices Ant task dives into any subdirectory within the decisionservices directory in an attempt to deploy that ‘Decision Service’. This is a minor glitch that requires a workaround. At this point, the pre-deploy and post-deploy targets that are part of the build file come in handy. A simple workaround is to temporarily remove the .svn directory like this:

<target name="pre-deploy">
    <!-- Add tasks here to be performed prior to process deployment. -->
    <!-- Temporarily remove the Subversion administration directory from
         the decisionservices as it will break the build
    -->
    <move file="${basedir}/decisionservices/.svn"
          tofile="${tempdir}/decisionservices/.svn"/>
</target> 

And when we are done, put it back:

<target name="post-deploy">
    <!-- Add tasks here to be performed after process deployment. -->
    <move file="${tempdir}/decisionservices/.svn"
          tofile="${basedir}/decisionservices/.svn"/>
</target> 

 

Using the Oracle BPEL Process Manager Client API for managing the environment

Oracle BPEL provides easy to use yet powerful versioning capabilities allowing you to run multiple versions of a process simultaneously. This way, existing instances of version 1.0 can finish while version 2.0 is used for new instances of that process. Upon deployment of a new version it likely needs to be marked as the default. The BPEL console provides screen functions for that purpose but that obviously is not very helpful for build and deployment automation. The Java client API (that is also used by the BPEL console) comes to the rescue. For marking a new process as the default revision, the following code will do:

public void markProcessAsDefault(String processId, 
                                 String revision) throws ServerException {
    if (locator == null)
        locator = getLocator();
    IBPELProcessHandle iBPELProcessHandle = 
        locator.lookupProcess(processId, revision);
    if (iBPELProcessHandle != null && 
        !iBPELProcessHandle.isDefaultRevision()) {
        iBPELProcessHandle.markAsDefaultRevision();
    }
}

The Client API is packed with other useful functions, e.g. for clearing the WSDL cache, something you need desperately during BPEL deployments. But the client API is also very useful for performing tedious management tasks like removing finished process instances from your BPEL system in order to keep the dehydration store lean and mean.

The pre-deploy and post-deploy targets in the Ant build files for a BPEL process also come in handy for performing these additional tasks. The following Java class extends Ant by turning the code snippet for marking a process revision as the default into an Ant task that can be invoked from the post-deploy target:

package nl.amis.soa.ant;

import org.apache.tools.ant.Task;
import org.apache.tools.ant.BuildException;

import nl.amis.soa.bpel.BPELBuildUtils;

public class MarkBPELProcessAsDefault extends Task {

  private String processId;
  private String revision;

  public MarkBPELProcessAsDefault() {
  }

  public void setProcessId(String processId) {
    this.processId = processId;
  }

  public String getProcessId() {
    return processId;
  }

  public void setRevision(String revision) {
    this.revision = revisi
on;

  }

  public String getRevision() {
    return revision;
  }

  public void init() {
    super.init();
  }

  public void execute() throws BuildException {
    if (processId != null && revision != null) {
        BPELBuildUtils bpelBuildUtils = new BPELBuildUtils();
        try {
            bpelBuildUtils.markProcessAsDefault(processId, revision);
        } catch (Exception e) {
            throw new BuildException("Process with id " + processId + 
                                     " and revision " + revision + 
                                     " could not be marked as default." +
                                     e.getMessage());
        }
    } else {
        throw new BuildException("Process not correcly identified; either processId or revision not given.");
    }
  }
}

 

Configuring Luntbuild

Now that the entire toolkit is installed and the Ant build scripts are created, it is time to configure Luntbuild. I simply created Luntbuild projects for each component in the SOA demo application.

Configuration is straightforward and I will not go into much detail here.. Important for building the BPEL processes is the correct setting of the environment variables. There is a simple trick to get there right: open a BPEL Developer Prompt window and grab the environment variables from there.

Notice that there is still an OBANT_CLASSPATH environment variable set .

For setting up dependencies between projects Luntbuild provides multiple strategies. I set up dependencies between the projects preserving the order that is outlined in the SOA demo application quick start guide and have each project trigger the next in order.

Since this blog post is not a Luntbuild configuration guide, I will not discuss the remaining configuration details here.

Stop talking, start building!

In the way I have arranged the dependencies starting the build for the BPEL processes and Web Services of the SOA Suite demo application requires the start of the build and deployment of the SelectManufacturer process.
Pictures speak louder than words:

Note that the ESB components in the demo application are missing, I created these with the help of JDeveloper. As indicated earlier, automating that step will be the subject of a following post.

Concluding remarks

So far so good: with modest effort I was able to set up my build environment and have reached the goals set for part 1. That confirms it is not hard to automate repetitive and tedious build and deployment tasks indeed. It also shows that Oracle has effectively leveraged industry-standard build and deployment techniques for its BPEL product. And I have not even touched on the possibilities of using tokens for deploying to multiple environments.
In-the-small, this blog demonstrated simple yet effective usage of technology to make your life easier. It applies to any software engineering effort but is especially relevant to SOA-based systems in which interoperation of loosely coupled components is important. Hope this helps you save valuable time and helps to provide your customers with consistent quality systems.