Oracle Enterprise Service Bus (ESB) – A First Impression (and it's not bad at all!)

You never get a second chance for a first impression, the shampoo ad warns us. So every software vendor is likely to be a little nervous about showing a 1.0 release of his software to a critical audience consisting of senior developers and – scary! – architects. It’s even worse to have to present them with a Beta Release. So what enormous bravery it must have been for Sandor to do a full one day workshop on the pre-beta release of Oracle ESB! No second chances for that first impression. However, and now you can breathe again, the first impression was most favorable! The Oracle ESB as it stands today – several months away from its foreseen production date – is not bad at all.

As you may have seen from the article by Harm (ESB – the new Oracle InterConnect), the ESB held up excellently through a series of non-trivial labs. We read and wrote files, read and wrote database tables, called out to Soap WebServices and had ESB Services be called by external entities. The ESB does content based routing, it does transformations, it has a series of pretty advanced adapters and in many ways it is very similar to the Oracle BPEL PM. The main discussions in this workshop were about when to use the Oracle ESB and when to use Oracle BPEL PM.

In this article I will try to describe that first impression. I will also go into some of the questions that were raised – about InterConnect, about Hub & Spoke vs. Bus, about error handling and so on. I will also describe some of the services we implemented in the ESB. And I hope to convey that feeling of: it’s a cool tool & technology that is fun to work with and it does some very valuable things that probably will help any organization.

Below you see how the ESB fits in the Oracle Fusion vision: part of Composition and Process Orchestration, living on top of the Enterprise Application Server with its embedded Messaging (AQ) infrastructure. Also note: no InterConnect in this picture. 

Oracle Enterprise Messaging (Service)

Packaged in the Oracle 10g Application Server is the OEMS – the Oracle Enterprise Messaging Service. Through OEMS, we can configure our Messaging Infrastructure – I believe per application or perhaps even per type of message (I have not seen all the details yet). Various methods for messaging – including various third party solutions – can be configured with OEMS. Our applications can transparently leverage the Messaging Infrastructure that has been setup in the Applications Server through OEMS, without knowledge about the actual messaging implementation. All messaging services are invoked through JMS (1.1) – the J2EE Java Messaging System API. Implementations of JMS setup through OEMS range from Oracle AQ to IBM MQ Series, from in memory to implementations from vendors like Tibco and Sonic MQ.

While performance, manageability, scalability and availability of the ESB depend on the transport layer, the functionality is completely transparent. We work with the ESB without any regard for the implementation of the transport layer. It is purely a deployment decision. Note that in the workshop we only used the in-memory JMS implementation (it was the light-weight approach), but it should have made no difference from a functional point of view if we had used AQ as our transport layer.

I nicked the following picture from the presentation SOA Programming Model at JavaOne 2006. It clearly illustrates the various layers in SOAs and indicates the position of an ESB in general:

Packaging and Availability

Oracle ESB will be part of the (core) Oracle 10gAS – including Adapters and Transformation Services. As such, it will not have its own price tag. Sandor’s suggestion that it could be called free is slightly pushing it, as the Application Server is everything but free. On top of the Oracle 10gAS, we can acquire the SOA Suite, with Oracle BPEL PM, Oracle BAM (Business Activity Monitoring) and WebServices Manager.

For organizations using a different Application Server infrastructure – e.g. BEA or WebSphere or JBoss – there will be another SOA Suite bundle, with BAM, BPEL and WSM complemented with ESB, Business Rules, B2B and JDeveloper + Plugins (for the design time).

Note that both ESB and BPEL design time – visual editors and wizard to create the ESB Services and BPEL Process definitions – will be released as normal JDeveloper plugins. That means that any normal JDeveloper installation can be used to plug these design time tools into and we no longer need special JDeveloper installs, such as is currently the case with the BPEL Designer.

No details about pricing were revealed at this time.

A first beta release – for participants in the SOA Suite beta program – is expected in a few weeks time. A production release for Oracle ESB as well as Oracle BPEL PM 10.1.3 is several months away (but very likely this calendar year and probably near the end of (a long and hot Indian) Summer.

Will the Oracle ESB replace InterConnect?

There have been questions about whether InterConnect would continue or cease to exist next to the ESB – just like we had such questions when Oracle first released the BPEL Process Manager. The answer is pretty straightforward this time: in due time – when it is production later this year – Oracle ESB will replace InterConnect. There does not seem to be any sense for organizations that have not worked with InterConnect before, to start doing so when the ESB is generally available. There may be some very specific scenarios defined by Oracle that preclude the use of ESB in very special conditions but in general: ESB with AQ as its transportlayer take over from InterConnect. And having seen both, I would say: thankfully so! Working with Oracle ESB proves to be much more intuitive and pleasant than using InterConnect.

Note that there is currently no migration tool from InterConnect to ESB. However, Sandor mentioned two customers who were migrating from IC to either BPEL or ESB (I cannot quite remember) and their experiences were very good: it proved relatively simple to migrate. There are also whispers of a real migration tool at some later point in time.

Key Components of ESB: Transformation, Adapters and Content Based Routing

The essential functionality of ESB infrastructures in general and the Oracle ESB in particular can be summed up to include at least

• Adapters to connect the ESB to external services (business connectivity)
• Business Data Transformation
• Content Based Routing
• Exception handling
• Distributed Implementation
• Synchronous and A-Synchronous Service Invocations

We have not actually seen it in action, but according to Sandor there is a facility called the Error Hospital for undeliverable messages (service requests). How those messages can be taken care of did not become clear yet.

Sandor suggested that we can have multiple ESB instances running in multiple instances of the Oracle Application Server. They would work against the same meta-data and run-time repository (where service instances are recorded). There could be a certain degree of stickiness – service only being executed in one or a selection of ESB instances. If one of the ESB instances was to die, the other could simply continue processing. What was not yet entirely clear is how in flight ESB Service executions could be recovered – if at all. I am not too sure about the relation between the underlying AQ infrastructure – if that
were used – and the ESB layer on
top of that.

Both Synchronous and A-Synchronous Service Invocations will be possible. We have not really addressed A-Synchronous Service Calls in this workshop. Note that ESB Services can be triggered by external events, picked up by pollers. They can also be invoked as standard SOAP WebServices. Sandor mentioned direct, local intra-JVM invocations from Java code. I do not know at this point how that would be done. Then there is the close integration with Oracle BPEL PM: in the 10.1.3 release there will be a new type of process step (or rather a macro-like piece of BPEL logic) that represents a call to an ESB Service. I hope to try that out very soon.

Adapters

The Oracle ESB comes with the same set of adapters – and design time wizards for configuring adapter-services – that we know and have come to love from Oracle BPEL PM. We can easily create Adapter Services within the ESB Services we create. That means for example that our ESB Service can be triggered by files arriving on the File system or an FTP server, by data being created in a database or by some event in Oracle Applications (so far I have not seen a counterpart for the ESB of the Email Activation Agent I have used with Oracle BPEL). It also means that our ESB Service can itself as part of the service execution call out to adapter services that write a file, perform a database operation, send a message to a JMS Queue or Topic, invoke an action on the Oracle E-Business Suite (aka Oracle Apps) or invoke a SOAP WebService.

Building an ESB Service using the Adapter Services is a breeze: a bit of drag and drop followed by a bit of wizard based configuration – you can do that! As Sandor pointed out at various moments during the workshop, some operations that can be a little awkward to set up as a BPEL Process can be much simpler when implemented as an ESB Services. A simple example would be – the first demonstration of the day – reading a file and writing that same contents (transformed or untransformed) to another file on another file system. Where the BPEL process may have taken several ASSIGN steps to copy data, the ESB service really is as simple as can be.

Business Data Transformation and Content Based Routing

One of the core ESB Services to be used in the ESB Service Definitions we create is the Router Service. The Router Service performs several tasks. First and foremost, it connects Sources (inbound service requests) to outbound services. Associated with each incoming connection can be one or more routing rules. The routing rules specifies that an incoming service request is passed on to a certain outbound service. It also allows us to specify a filter: using an XPath expression we specify if and when a Service Request should indeed be processed to that outbound service. Defining the Filter Expression is done using the XPath Expression Builder you probably know from Oracle BPEL PM. It is really simple and hardly requires XPath knowledge. Finally we can associate a Transformation with that passing through of the service request: the incoming XML request can be transformed to the format required for the next service. The Transformation Map is specified through the Mapper editor, a visual editor of what is basically an XSL-T stylesheet. We can easily connect source and target elements in this editor, adding XSL-T functions to perform filtering, concattenation and splitting and other types of transformation on the data.

The CBR consists of Expression Rule that determine to which outbound service a service request is passed on, based on XPath expressions evaluated against the contents of the Service Request. The filter specifies go or no go for a specific outbound service. Since we can have multiple Routing Services and perform transformations in each Routing Service, we could have one RS set up with fairly complex transformation logic followed by a RS with fairly simple Filter Expressions. I have not yet formed a complete picture of how we can best organize the ESB Service Definitions. I do have a feeling that as soon as things start getting complex, we should probably resort to a BPEL Process rather than an ESB Service.
Brainstorm/Discussion

Having worked all day with the ESB, a lot of questions were answered. Yet I think that as many new ones were raised. The software works surprisingly well, the concepts are pretty straightforward, implementing quite interesting scenarios proved easy enough and the integration with Oracle BPEL PM is cool. And now we wonder: how does it all fit together in the real world? What will we do in the ESB – if we can also do it in Oracle BPEL PM? How much logic is to be built into the ESB Services themselves – and what will we do in external services and rule engines? Should we only call ESB Services from our BPEL Process and never invoke Partner Links directly? Can we combine RouterServices in our ESB Service definitions that do smart XSLT transformations to implement advanced and unintelligible routing rules or should we leave this to the BPEL Process?

The questions are interesting as is the discussion. Clear cut answers are hard if not impossible to come by. The world of SOA, BPEL and ESB is too young for definitive answers. Which makes it all the more interesting to us, archictects and technical specialists.

Service (URL) Virtualization

One of the main tasks any ESB could and should perform is Service (URL) Virtualization. The ESB exposes a number of Services. These Service are not implemented as such by the ESB. The ESB knows where to find these services and how to invoke them. It decouples the service providers from the service consumers. These consumers only know about the services published by the ESB and only need to know how to approach the ESB. If any of the underlying services is changed or relocated, the ESB may need to change one of its Adapter Services or Routing Services, but the Service Consumers would not need to know about this. The ESB then becomes the Enterprise Service Broker – a Hub for Service Requests.

One of the topics for discussion in the final part of the workshop was the question whether we should adopt as a guiding principle that no service request is made directly, for example from BPEL Processes. Instead, each Service Request should be addressed to the ESB that turns it into the proper Service invocation – possibly depending on the content of the Service Request – ,with the proper input parameters as required for the service implementation that gets called.

It sounds tempting: its clean, its straightforward and relatively simple. However, it adds some overhead compared with the current way of invoking external services directly from the BPEL process itself. So how do
we strike a balance between architecturally sound and pragmatic. It will not surprise you that the discussion was undecided at this point.

ESB and BPEL

One question that kept going round was: how will functionality be assigned between ESB and BPEL. They work together very well: BPEL can invoke ESB Services as easy as other BPEL processes and perhaps even easier than Partner Links through Adapter Services. The ESB is designed to be at least five times faster than the BPEL PM at similar tasks. Of course BPEL PM has many additional facilities in terms of monitoring, audting, recording history etc. ESB Services can be tweaked at run-time (routing rules as well as transformation mappings can be changed on the fly. Creating an ESB Service Definitions seems slightly easier than creating corresponding BPEL Process Definitions.

One thing is clear – we will be looking for the exact balance between these two for some time to come.

Resources

The software is not yet available to the general public (especially not for those living in certain countries).

Business White Paper: http://www.oracle.com/technology/products/integration/esb/pdf/ds_esb_v10_1_2.pdf