The Composite Service engine – The future role of BPEL – filling the niche between Enterprise Service Bus and BPM engine?

One question that keeps me awake at night is not exactly the position of BPEL within SOA in time to come. However, during the day, that question does keep popping up in my head. In today’s SOA technology landscape, it is not a trivial question. In fact, I heard on Oracle ACE Director quote as having declared: "BPEL is dead", referring to BPA/BPM as the extinguishers. While I am not prepared to make such extreme statements, the position of BPEL is one that justifies some deliberations.

A few years back, SOA in my mind was almost synonomous to BPEL. In part because my world tends to be defined to great extent by the Oracle product portfolio as that is the scope in which I usually work. (that has been changing to in the last few years by the way). But even within that Oracle portofolio, the SOA Suite has become so much more than the BPEL engine that started it all off. In 2006 Oracle started carefully introducing its ESB. Later on it introduced the BPA Suite – not yet a run time engine, but a clear focus on Business Process modeling nevertheless.

The recent acquisition of BEA has accelerated the process enormously. With BPM Studio (pka Fuego) Oracle now has a run time engine for business processes – one that sits much closer to business analysts than BPEL ever did or will. And one that is more flexible when it comes to designing and running processes. On the other end of the spectrum, the ALSB now called the Oracle Service Bus provides a mature, proven Enterprise Service Bus. One that has a much broader scope than the Oracle ESB possessed.

Until quite recently, Oracle BPEL PM was seen as the component for designing and executing Business Processes (hence the BP in its name, not an Oracle invention obviously), publishing Services that contain complex business logic that include orchestrated service calls, exception and compensation handling, correlation and (potentially) long running state. However, due to lacking functionality (and initially a lack of product altogether) in the ESB, more simple enrichment, parallel execution and asynchronous patterns were also frequently implemented in BPEL. While the initial assumption that BPEL would also be used by Analysts for describing processes proved mistaken, as BPEL is far too technical in nature with all the XML involved, use of variables, schema definitions and exception handling, BPEL was still used to design business processes. Using the Human Workflow Service offered along with BPEL PM in the SOA Suite, processes can contain both (system) service oriented logic and human activities.

Oracle BPA was introduced at the business end of things. With BPA, the business analyst describes among many other things the business processes, for example in terms of swimlanes, decision points, process-steps/activities and flows between the latter two using BPMN – the open standard for Notation of Business Process models. This business view can be generated/converted into a BPEL Blueprint which is the starting point for a BPEL developer. BPA is used for design, BPEL for implementation. There is a substantial gap between the two – the BPA (BPMN) terminology has some aspects that do not translate one to one into BPEL constructs, most notably perhaps the goto. Note that the transition from BPA to BPEL is necessary as BPA does not have it own run time engine. Also note that BPMN was created to be the business modelling, analyst level language for the definition of the process that was to be complemented by executable languages to run the process. BPEL (Business Process Execution Language). With BPA and BPEL we have it all together – for design, implementation and execution.

With the Oracle BPM stack on the process, workflow oriented end and the Oracle SB (pka ALSB) on the other end of the SOA spectrum, it would seem that many things we used to do in BPEL will now be easier, more appropriately done in either BPM or the OSB. BEA BPM has its own runtime engine – turning BPMN (or the BPM Studio flavor of BPMN with BPM Studio specific extension) into an execution language as well as a design language. The latter (OSB) supports much more complex flows than OESB did. It deals with Asynchronous services, for-each iterations, variable manipulation, enrichment, more involved branching logic, some exception handling – many things we previously had to resort to BPEL PM for.

In a picture taken from an OOW presentation:

OSB is not meant for very long running processes (or should I say flows or services?) – very long being longer than a few seconds. Nor is it meant to carry a lot of state. Longer running processes and processes with a lot of state are probably more suitable for BPEL than for OSB (note: in the near future, BPEL PM is likely to start using Coherence for dehydration purposes, making management of state for large numbers of long running, concurrent process instances transparant and manageable/scalable). Processes with very may service calls and quite complex message manipulation, support for compensation of already successfully completed operations etc are also candidates for BPEL.

The next picture – somewhat inspired by a slide by Lonneke and Ronald in the Oracle vs BEA shoot out – sums up my current understanding (October 2008) of where we are (going):

One consideration to bear in mind: not every organisation will have both SOA and BPM suite. And may therefore use BPEL instead of  BPM for the design and implementation of processes. Nothing wrong with that. However, BPEL seems more for technical processes and developers while BPM (also) appeals to business oriented processes and analysts.

Oracle presents the following picture for the 11g SOA Suite (Service Fabric):

It lacks the BPM engine. Which can be quite an important difference. The picture does not really explain the distinction between OSB and BPEL. They will work closely together within the same run time environment.

For the somewhat longer term future, Oracle announced that BPM and BPEL will share the same runtime engine. Whether the process is described from the BPM (Studio) side of things or whether BPEL was used does not really matter in the run time environment. On the hand it seems logical – running a process is what it should do, no matter how the process was described – at run time it will always be a combination of service calls, data manipulation, simple evaluations and decision logic. However, why have a BPM run-time at all?

Today I came across the following picture which takes it even a step further, suggesting a design time alignment between BPM and BPEL – including the ability to do two-way conversion between the two:

As BPMN was originally supposed to be complemented by an execution language such as BPEL anyway, this approach this probably more logical than just having the same run-time engine. Having a BPM run-time engine in the first place seems somewhat odd – at least when the roundtrip engineering between BPM(N) and BPEL is very smooth, you would expect to always translate the BPM model into a BPEL process that is then either further refined or immediately executed.

Summary

It is still interesting to see how things evolve. How close the integration between BPEL and BPM on one end and OSB on the other will be for example. I feel that BPEL is primarily the component for composing composite services – services that call other, more fine-grained services, handle logic and exceptions and asynchronicity between those service calls. Services that may be around for longer periods (that’s when a service may become a process?), carrying some state and being available for correlated additional calls. And that can listen to events taking place that influence the already running service/process instance.

Maybe whenever you are in doubt whether to call something a process or a service, it should be a BPEL process (service)?

I would be interested to hear your thoughts on the matter!