Oracle SOA Suite: further Build and Deployment Automation for Enterprise Service Bus components

Some time ago I wrote an article on SOA Suite Build, Deployment and Test Automation. A section of the article was devoted to automating builds and deployments for Oracle Enterprise Service Bus (ESB) components. Not covered was deployment of Services to multiple environments as is usually done throughout the development and test process of a system. This blog post starts where the article stopped: it describes build and deployment automation for an ESB System in a way that it can easily be deployed to multiple environments that support the system development life-cycle.

Support for build automation in SOA Suite

It seems that Oracle has not gone to extremes in providing support for build automation for SOA Suite. Sure, from within JDeveloper it is easy to compile and deploy BPEL and ESB components to diverse environments. But I would love to see Oracle move (further) away from the use of JDeveloper for this purpose as it stands in the way of automating the build, deployment and test process. Oracle should embrace Ant or Maven and make sure that the exact same scripts are used from within JDeveloper as well.
The importance of a streamlined and manageable build and test process is illustrated by the comments on the SOA Suite forum that is hosted on Oracle’s Technology Network. For example, see this thread for ESB deployment automation. And in some cases this even results in a heated debate .

Setting up a streamlined build and deployment process is of vital importance for any system development effort. This is especially true for a SOA implementation in which you need to ensure that a robust system is constructed from various loosely coupled services. It may require some initial effort and getting used to, but in the end this approach will save you valuable time and help provide your customers with consistently high-quality systems.

Sample Scenario

For demonstration purposes I have created an extremely simple scenario: an ESB System takes in a text string that it passes on to a synchronous BPEL Process. The BPEL Process and the ESB System do not manipulate the text; in fact, nowhere in this scenario is any intelligence to be detected. The following picture provides an overview of the BPEL Process:

And here is the equally simple ESB System that does no more than route the message to the BPEL Process:

The default XML schema that is used for the BPEL process is also used as input for the ESB Routing Service that is defined in the ESB System:

Use WSDL URL’s and ESB SOAP Services as the target for the ESB System

Note that the HelloWorld BPEL Process that is the target Service in the ESB System was not selected using the ‘Browse Target Service Operation’ dialog in JDeveloper that is shown in the following screenshot:

Constructing the ESB System that way would have resulted in the following picture of the ESB System. Notice the subtle difference in comparison to the overview picture that was appeared earlier:

And there are more differences below the surface, e.g. in the generated .esbsvc file for the Service. Using the browser for defining the target service makes us run into deployment problems as well as runtime issues; hence we refrain from doing that. We always use the WSDL URL for the intended target service and create a SOAP Service in the ESB Designer as shown in the following screen shot:

Build and Deployment Automation prerequisites

Oracle’s Doug Gschwind created a badly needed Ant extension that supports build and deployment automation for ESB Systems. It may be downloaded from OTN. The approach outlined here builds on Doug’s work.
First job is to install Doug’s Ant extension. Check the included manual and the OTN Forums for assistance if required. We use Ant 1.7 for running it. An Ant build script has to be added to every ESB System. Place it in the ESB Project’s directory. The template we use in our projects is shown later. It must be adapted to contain the global unique identifiers (or guids) for the artifacts in the ESB System. This will only take a few minutes.

Build and Deployment Process Steps

With the Ant extension and the build.xml script in place the deployment process for an ESB System involves the following steps:

1. Deploy the ESB System in an environment of your choice. An initial deployment is required in order to generate a Deployment Plan. Typically, the development environment is used for this purpose.
2. Adapt the Deployment Plan for deploying the ESB System to any other environments that support the system development life-cycle. Essentially, this means changing the WSDL URL’s so that these match up with the environment-specific Service URL’s.
3. Use the adapted Deployment Plan to roll out the ESB System to the environment of choice.

As we will see, these steps can be highly automated so that deployment of an ESB System to any environment is reduced to a single command.

The Ant Script for Automating the Process

Without further ado, here is the Ant build.xml script that we add to ESB Systems:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<project name="ESBDeploymentProject" default="usage">

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Properties
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

   <property environment="env"/>
   <property name="esbProjectToDeploy" value="${basedir}"/>
   <property name="deploymentPlanFilename" value="${esbProjectToDeploy}/deploymentPlan.xml"/>
   <property name="targetDeploymentPlanFilename" value="${esbProjectToDeploy}/targetDeploymentPlan.xml"/>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Environment specific properties
   - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - -->
   &lt
;property file="${env.SOA_DA_HOME}/environment.${param.target.env}.properties" />

   <target name="param-target-chk" unless="param.target.env">
     <fail message="Target deployment environment parameter not set" />
   </target>
   
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Import, to enable the custom ESB Metadata Deployment ant tasks ...
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <import file="${env.SOA_DA_HOME}/lib/ESBMetadataMigrationTaskdefs.xml"/>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - ESB Deployment Automation
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <target name="DeployESBProject" depends="param-target-chk">
      <deployESBProjects
	    esbMetadataServerHostname="${soa.suite.hostname}"
		esbMetadataServerPort="${soa.suite.port}"
        userName="${soa.suite.admin.username}"
        password="${soa.suite.admin.password}"
	  >
	  <esbProject directory="${basedir}"/>
	  </deployESBProjects>
    </target>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - List the GUID's for the UndeployESBEntities target e.g.:
	    <system guid="8D61C3F0871111DB8F2675C60E6C31C6"/>
	    <serviceGroup guid="0EB5F380896111DBBFBC9530C01627AC"/>
	    <service guid="0547F370841611DBBFCF2D9BF80323FA"/>
	    <serviceGroup guid="B90E6B70895F11DBAF1483EEF470B835"/>
	    <system guid="A62C91C1841511DBBFCF2D9BF80323FA"/>
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <target name="UndeployESBEntities" depends="param-target-chk">
      <undeployESBEntities
	    esbMetadataServerHostname="${soa.suite.hostname}"
		esbMetadataServerPort="${soa.suite.port}"
        userName="${soa.suite.admin.username}"
        password="${soa.suite.admin.password}"
	  >
	    <system guid="AF254B90559411DCBF88696C2BB1E7B8"/>
	    <service guid="25E5FDB0559511DCBF88696C2BB1E7B8"/>
	    <service guid="F5A618A0559A11DCBFBF4119724B0CE3"/>
	  </undeployESBEntities>
    </target>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Metadata Promotion to different ESB Metadata Servers (environments)
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <target name="ExtractESBDeploymentPlan">
        <delete file="${deploymentPlanFilename}" verbose="true"/>
        <extractESBDeploymentPlan
           sourceDir="${basedir}"
           deploymentPlanFile="${deploymentPlanFilename}"/>
    </target>

    <target name="ModifyESBDeploymentPlan" depends="param-target-chk">
      <xmltask source="${deploymentPlanFilename}" dest="${targetDeploymentPlanFilename}">
        <replace path="/deploymentPlan/systemDeploymentPlan/properties/property[@name = 'Host']/@value" withText="${soa.suite.hostname}" />
        <replace path="/deploymentPlan/systemDeploymentPlan/properties/property[@name = 'Port']/@value" withText="${soa.suite.port}" />
      </xmltask>
      <replaceregexp file="${targetDeploymentPlanFilename}"
	                 match="http://[^/]*/"
	                 replace="http://${soa.suite.hostname}:${soa.suite.port}/"
                     byline="true" />
    </target>

    <target name="DeployESBSuitcase"
	        depends="param-target-chk, ExtractESBDeploymentPlan, ModifyESBDeploymentPlan, UndeployESBEntities">
      <deployESBSuitcase
        esbMetadataServerHostname="${soa.suite.hostname}"
        esbMetadataServerPort="${soa.suite.port}"
        sourceDirectory="${basedir}"
        deploymentPlanFilename="${targetDeploymentPlanFilename}"
        forcedDeployment="false"/>
    </target>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Usage
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <target name="usage">
      <echo>
========   Environment   =====================================================
  SOA_DA_HOME      = ${env.SOA_DA_HOME}
  CLASSPATH        = ${env.CLASSPATH}
  ANT_HOME         = ${env.ANT_HOME}
========   Properties    =====================================================
  basedir          = ${basedir}
  SOA Suite server = ${soa.suite.hostname}:${soa.suite.port}
  OC4J instance    = ${soa.suite.oc4j.instancename}
==============================================================================
      </echo>
	  <exec executable="ant" dir="${basedir}" vmlauncher="false">
	    <arg value="-projecthelp"/>
	  </exec>
    </target>

</project>

The only change to make this template work for the ESB System from the scenario was to look up the guid’s for the System and the Services in the project and put them in the proper place, as indicated in bold.

Note that the build and deployment process is environment specific:

• We check if the parameter ‘param.target.env’ is set.
• We use a set of environment-specific properties files (environment.${param.target.env}.properties) to drive ESB and BPEL deployments alike.

Running the DeployESBSuitcase target performs the following tasks for deploying the ESB System to the target environment:

• First the Deployment Plan is extracted.
• Subsequently the Deployment Plan is modified for the specific target environment using the ModifyESBDeploymentPlan target.
• Any existing installation of the ESB System in the target environment is removed by executing the UndeployESBEntities target in order to ensure a clean install.

Modifying the Deployment Plan

The generated Deployment Plan for this service looks like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<deploymentPlan>
    <systemDeploymentPlan qname="HelloWorldESB">
        <properties>
            <property value="" name="Fax"/>
            <property value="" name="Mobile"/>
            <property value="soa10133-vm" name="Host"/>
            <property value="" name="Phone"/>
            <property value="" name="Pager"/>
            <property value="" name="Email"/>
            <property value="80" name="Port"/>
        </properties>
        <serviceDeploymentPlan qname="HelloWorldESB.HelloWorldBPEL">
            <parent type="system" qname="HelloWorldESB"/>
            <properties>
                <serviceDefinition>
      <property value="http://soa10133-vm/orabpel/default/HelloWorld/1.0/HelloWorld?wsdl" name="wsdlURL"/>
                    <property value="http://soa10133-vm:80/event/HelloWorldESB/HelloWorldBPEL" name="soapEndpointURI"/>
                </serviceDefinition>
            </properties>
        </serviceDeploymentPlan>
        <serviceDeploymentPlan qname="HelloWorldESB.HelloWorldRS">
            <parent type="system" qname="HelloWorldESB"/>
            <properties>
                <serviceDefinition>
                    <property value="HelloWorldESB_HelloWorldRS.wsdl" name="wsdlURL"/>
                </serviceDefinition>
            </properties>
        </serviceDeploymentPlan>
    </systemDeploymentPlan>
</deploymentPlan>

We added the ModifyESBDeploymentPlan target to the build.xml file for modifying this Deployment Plan in order to turn it into an environment-specific one. Benefiting from the well-formed XML file the Deployment Plan is, we use the XMLTask Ant extension for changing the Host and Port properties. Moreover, we apply a regular expression for making the WSDL URL of Services environment specific. The regular expression “http://[^/]*/” that we use for matching looks for any pattern between “http://” and the first forward slash that follows. This can be changed into something less generic if everyone in the project sticks to a standard like “http://host.domain:9700/”.

We added the XMLTask task definition to the ESBMetadataMigrationTaskdefs.xml file that is provided in Doug’s original package to make it look like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - The import of this ant build file, by another ant build file, enables the
   - use of the custom ant tasks present in ESBMetadataMigration.jar
   - Doug Gschwind
   - 12 Dec 2006
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

<project name="ESBMetadataMigrationTaskdefs">

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - PROPERTIES, Subject to the installation environment
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <property environment="env"/>

<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Custom ant task definitions, to enable import.
   - This section should be treated as immutable upon installation.
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->

    <dirname property="imported.basedir" file="${ant.file.ESBMetadataMigrationTaskdefs}"/>

    <taskdef resource="oracle/tip/esb/client/anttasks/antlib.xml">
        <classpath>
            <pathelement location="${imported.basedir}/ESBMetadataMigration.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/commons-httpclient-3.0.1.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/xmlparserv2.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/commons-logging.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/commons-codec-1.3.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/oraesb.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/activation.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/jaxb-api.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/jsr173_1.0_api.jar"/>
            <pathelement location="${env.SOA_DA_HOME}/lib/jaxb-impl.jar"/>
        </classpath>
    </taskdef>
    
    <!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   - Define Ant XML Task for manipulation of the deploymentPlan file
   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
    <taskdef name="xmltask" classname="com.oopsconsultancy.xmltask.ant.XmlTask">
      <classpath>
         <pathelement location="${env.SOA_DA_HOME}\lib\xmltask-v1.15.1.jar"/>
      </classpath>
    </taskdef>

</project>

Enjoy building and deploying Oracle SOA Suite ESB Services!