Want to know more on Oracle BPEL Process Manager and in particular about the upcoming release with the JDeveloper plugin? This article gives an overview as well as some screenshots of exactly that. On Thursday January 27th – we had another AMIS Query, this time on Oracle BPEL Process Manager. For this session, we have joined forces with Oracle. Sandor Nieuwenhuijs (Oracle’s Senior Principal Product Manager for Oracle BPEL Process Manager) presented and demonstrated Oracle BPEL Process Manager, both the current offering with an Eclipse Design Time plugin as well as the beta preview of the coming release that has a JDeveloper plugin. The audience was largely technical in nature. About half had (also) worked with Eclipse, the vast majority were JDeveloper users – a novelty for Sandor who usually presents to Eclipse minded audiences.
Handouts for this session can be downloaded here: Oracle BPEL Process Manager – handouts for AMIS Query, January 27th. Note that this does not include the software handouts that all participants received: the not yet available Beta for the Oracle BPEL PM Design Time plugin for JDeveloper! Bound by a virtual non-disclosure agreement, we are not allowed to distribute this software or tell others too much about it.
An excellent write-up of many of the important stuff of this session was written by Andrej Koelewijn, one of our most-distinguished guests, in his weblog: Oracle BPEL jDeveloper plugin
BPEL – Business Process Execution Language
Sandor started with an introduction of some of the concepts and terms around BPEL, including BPEL itself. He made it clear that to him there is not a large and meaningful distinction between Workflow (UI oriented), Business Process Management (BPM, no user interaction) and BPEL (may have both). Data Integration has a somewhat different focus and approach but shares a blurry border with BPEL and the other process integration oriented domains. The concept of reusing services and assembling new services from existing services is nothing very new. Libraries, collections of reusable components, CBD have all been in vogue for many years. Somewhat new with BPEL is the scope of the Services (typically though not necessarily on a grander scale), the fact that BPEL bridges technologies, but most importantly: BPEL is standards-based (XML, XSLT, WebService, SOAP, BPEL itself,…).
BPEL is the standards-based successor/competitor to proprietary implementations for process integration such as Tibco and offerings from WebLogic, Oracle, IBM etc. BPEL is the sequel to over five-year old unrelated initiatives from IBM (WFSL) and Microsoft (XLang). BPEL is currently backed by IBM, Microsoft, SUN, BEA, Oracle. There are actual BPEL implementations from IBM, Oracle, Open Source teams and planned products from BEA and to some extent Tibco (only import/export functionality).
Sandor describes many aspects of Services in general and Services to be integrated into BPEL processes in particular. While many Services, especially the ones that cross the boundaries of organisations, may be implemented as SOAP based WebServices, some or probably even most will not. Oracle BPEL Process Manager allows binding of all kinds of Services through much simpler ‘protocols’ than full-fledged SOAP WebService implementations. You will still need as WSDL for every service, but not necessarily a Http and SOAP based implementation. Later in this post some more on this Binding Framework and the Adapters Oracle BPEL provides for binding Services implemented in various technologies such as PL/SQL, Java, Files (?!), JMS and Oracle Apps. The Apache WSIF framework plays an important role in this Binding architecture. WSIF is shipped along with Oracle BPEL. With this type of Service Invocation, the transactional context can be maintained and controlled, for example through JTS. With normal WebServices, this is not the case; the best you can hope for is a WebService that offers different phases of submitting data: “ok, this is what I would like but wait for my confirmation before committing the change” followed by “ok, now I am sure” or “Please undo my previous request, I cannot complete the transaction”. In addition to the above, Oracle works closely with iWay, a firm that provides adapters for many different products and platforms such as Siebel, Baan, Oracle JDEdwards – as Sandor gloatingly spoke – Oracle PeopleSoft, etc. The iWay adapters will be available for binding services offered by these products and platforms into BPEL processes.
BPEL seems to make sense to at least consider whenever a process is implemented by pulling/wiring together (existing) services. As stated before, these services can be implemented in various ways and in different technologies. A BPEL process is published itself as a WebService by the BPEL runtime engine. Implementing a BPEL process to invoke services and combine the individual results through a logical flow into a new end-result may have some overhead in terms of performance and initial development time compared to simply writing program code that natively invokes all services (provided such is possible given the possibility for these services to be implemented in different technologies). However, the true gain of BPEL lies in standardization, maintenance, the generic runtime engine with many management facilities, decoupling of applications and bridging of technologies. The upcoming WebService Gateway – in Oracle AS 10g 10.1.3 – will add additional functionality to the runtime engine for WebServices, such as Security and Monitoring – for example of Service Level Agreements (SLA).
Sandor mentioned a large Belgian Tele-Communications company – soon to be a reference site – that implemented a BPEL process recently that spanned 17 different systems in many different technologies. On day one of the production rollout, this process was executed over 15.000 times with better than expected – and required – performance.
Oracle BPEL PM Architecture
Some remarks on the Oracle BPEL Process Manager Server (runtime!) Architecture:
The BPEL Process Manager is a J2EE application that runs in a J2EE Container – such as OC4J, JBoss etc – leveraging many different J2EE facilities. Most BPEL process are initiated by WebService calls that are received through Http; some process are more or less auto-invoked by the reception of a notification such as the arrival of a file on some file system. Note that a BPEL Process is the business process as you have defined and designed it and generally described in the BPEL XML Document. Whenever the process is invoked/called/executed, a new Process Instance is instantiated. For most BPEL processes, you will have many different instances – some in parallel i.e. running at the same time. We hope to receive multiple Orders or Requests for Proposals or Bookings for our Hotel-rooms/Rental Cars/Consulting Services at the same time.
Execution of a BPEL Process Instance typically involves calling (Web)Services, waiting for responses, performing some logical processing on the results, calling more services and calculating a final result. Some of the Services that get invoked during a process execution may be User Tasks (or workflows) that require human intervention. Each process instance is monitored and logged on at each stage of its execution. All information about individual process instances – their current status and each step along the way including all messages exchanges with external services – is available through the BPEL Console, a browser based web application. Through the Console, test-instances of processes can be started and even a load-stress test can be run. A statistical report of that stress test is available, illustrating the total load and time per process execution. The console also provides debug information: the console displays for the debugged process instance the BPEL process design and the values of all variables at each stage of process execution.
Oracle BPEL Process Manager saves information about process instance execution – and even the entire process instances themselves along with all variable-state – in a database (that supports JDBC). This process of svaing all process-state is called ‘dehydration’. Of course Oracle assumes that this database will be an Oracle 10g RDBMS but in fact BPEL PM ships with schema definitions for Oracle, MS SQLServer and other databases. Iiteresting to note that BPEL PM itself determines when to save state to the database – you cannot really influence that decision.
Of course the dehydration to the database has two major advantages – scalability and reliability:
- When a BPEL server engine instance crashes, the process state is not lost: it is available in the database and can – largely – be reinstated from there
- Related to this first benefit: when we want to scale up our BPEL Process Server, we can implement multiple BPEL PM instances on several Application Servers and Load Balance between them. A process instance can start life on one BPEL server instance, be dehydrated to the database and hydrated again in another instance to complete execution
Tough Questions
Many questions were raised during the session and among them the Top 3 most frequently asked questions:
- How does Oracle BPEL relate to Oracle Workflow?
- Is there still a future for Oracle ProcessConnect?
- What is the relation with InterConnect?
Sandor did his utmost to escape answering some of these by embroidering on often these questions were asked and how complicated the answers. However, it seems that the long and short of it (not Sandor’s words but my interpretation of them):
Oracle Workflow will continue to exist because of its large installed base through Oracle Applications; Apps makes heavy use of Oracle Workflow. For that reason alone, we can expect Workflow to have an extended lifespan. However, Oracle Workflow is not standardized – although there are plans apparently to support BPEL in Oracle Workflow ?! – and primarily targeted at PL/SQL processes. It does not seem wise if you do not already have current investments in Oracle Apps, Oracle Workflow application or skills, to embark on Oracle Workflow based projects. Oracle BPEL PM is the better option.
ProcessConnect has no future as it used to be. ProcessConnect really consists (consisted?) of two parts: BPM and B2B. BPM (Business Process Management) is a goner, replaced by Oracle BPEL PM. B2B provides functionality to communicate from various sources – such as AQ, EJB, Tibco, InterConnect – with Business Partners through B2B protocols such as EDI and RosettaNet. This B2B component will live on as part of Oracle Integration – as of release 10.1.2 that is already available from OTN.
InterConnect is the other major component on the Integration CD. Sandor sees BPEL and InterConnect living and working together. InterConnect is typically more direct, within the organisation and bridging technologies to a somewhat lesser extent – based on AQ – and also less standards-based. Pure data integration – e.g. the Orders system must ensure that all new customers are fed into the main CRM system – will be the area where InterConnect takes charge. When data integration takes place in more complex processes with several systems interacting and logical flows defined, BPEL seems the more obvious choice. Of course the Services invoked by a BPEL process can very well interact with InterConnect, so the two worlds are close together.
Release schedule
The production release currently available is BPEL Process Manager (Server, runtime) 2.1.1. Patch release 2.1.2 will come out very soon. Interesting new (?) functionality in the 2.1.1 BPEL Designer is the UNDO function (multiple steps).
The development team is currently cranking out release candidate releases every week. Before too long, one will be considered good enough to make it onto OTN as Sneak Preview. Sandor refered to it as Beta 2 and suggested it is only a matter of weeks before it is released. Internally they are working and demonstrating with what you could call a Beta 1. All participants of this session received a CD with the current state of affairs, a Beta 1.5 somewhere between Beta 1 and 2.
The Production release of BPEL 10.1.3 will be part of Oracle 10g Application Server 10.1.3 and along with JDeveloper 10.1.3. Sandor mentioned April a couple of times but hastened to stress that was not a commitment.It’s my own gut feeling that it will be somewhat later than April, closer to June – given all the work going on for example in JDeveloper. We will see.
Its the ambition of the Product Development team to keep the JDeveloper and Eclipse plugins more or less in synch. There are no plans whatsoever to abandon the Eclipse plugin, as some Oracle staff have suggested. The world is massively moving towards Eclipse, and as much as the Oracle community my like JDeveloper, in the real world it is only a very small contender in the Java IDE arena. JDeveloper is positioned to increase sales of Oracle RDBMS and primarily the Oracle 10g AS. Oracle BPEL does not have to increase JDeveloper sales but instead should also contribute to Oracle Application Server. (Sandor mentioned by the way that recent research indicates that Oracle’s Application Server is now a confident third in terms of marketshare, behind BEA WebLogic (2nd) and IBM WebSphere (1st). Note by the way that Oracle BPEL Server is available in distributions for Oracle AS (OC4J), JBoss (open source, free!) and WebLogic. Internally there is one for WebSphere as well, but not freely available for download and not heavily marketed.
Miscellaneous
Release 10.1.3 of Oracle BPEL will have documentation – a reference manual. That is new since up to now the documentation (only) consisted of samples. For almost every task and the more common situations, the product provides sample projects that illustrate how to implement required functionality. These samples often provide piece of process design that can easily be incorporated in the processes you design yourself.
There is book on BPEL that even has some references to Oracle BPEL: Business Process Execution Language for Web Services : BPEL and BPEL4WS by Matjaz Juric, Benny Mathew, Poornachandra Sarang; Publisher: Packt Publishing (October, 2004) ISBN: 1904811183
We briefly discussed the relationship between BPEL Process Design and the more conceptual level of describing Business Processes, such as UML Activity Modeling. Sandor did not see a real benefit in deriving BPEL process designs from the UML models. Oracle has no direct plans in that direction and he believed IBM at one time did something in that direction but has abandoned that project. Andrej suggested the other way would be useful: derive (capture) UML Activity Model or other types of Process Design from BPEL process definitions; this would allow for conceptual level over-all process blue-prints that were based on or derived from the actual process design and implementation. Sandor thought this an interesting suggestion.
The output of the BPEL Design Time is basically just an XML Document for each BPEL Process. So the BPEL Design Time is nothing more and nothing less than a UI to more productively and with less errors produce these XML Documents. It also means that by and large these BPEL Processes can be fed into other BPEL Run Time engines besides Oracle BPEL Process Manager.
Variables in a BPEL Process are all XML Documents. Variables can become quite voluminous; the data exchanged with all invoked services can amount to many Kbs or even Mbs. All global BPEL Process variables are dehydrated to the database and kept in memory when hydrated. By defining variables at scope-level within a BPEL process, the lifetime of those variables is limited to that scope. When the process execution leaves the scope, the scope-level variables are discarded and no longer take up memory and database resources.
Oracle BPEL PB allows the execution within process instances of in-line Java code. This code is included in the BPEL process using the BPELX standard extenstion tags in the BPEL language. It is a standardized way for doing non-standard things. Sandor did not recommend using in-line Java. It makes the BPEL process less portable, less standardized. Exchanging variables from the BPEL process with the in-line Java can be somewhat cumbersome. With the simple and direct binding of Java services, the need for in-line Java is very small in his opinion. IBM together with BEA by the way is working on an initiative they call BPELJ: BPELJ is a combination of BPEL and the Java programming language allowing the two languages to be used together to build business process applications. It is not entirely sure how standardized and portable their BPEL processes are. Google for BPELJ and IBM and you get a substantial number of hits offering more details on BPELJ.
Then there is JBI. From an article on IT Manager’s Journal:
When it comes to Web services integration, Java application vendors have elected to go the way of diversity.
BEA and IBM will use a language called BPELJ, which the two companies have been co-developing since March 2004, while JBoss, Novell, Oracle, Sun, Sybase and 17 other companies released a public version of a Java specification, JBI, which they have been working on since March 2003. JBI also is known as JSR 208. While BPEL itself orchestrates the interactions among Web services, these other two efforts seek to provide an intermediate translation layer that makes other applications appear to be Web services so that they, too, can be orchestrated in these chained transactions. BPELJ does this as an alternate scripting language, and JBI does this by defining a new standardized architecture for plugging components into the Java platform. BPEL engines could be one of many components that would be plugged into that architecture.
The BPEL Console currently has a dependency on Inter Explorer 6.0 (+), because of a number of ActiveX controls – such has MSXML and MSJScript – that are used. Sandor was not proud admitting this and work is underway to fix this dependency. Within Oracle for example, Mozilla FireFox has a 30% browser-marketshare. A dependency on IE is simply not acceptable. However, as it stands, the Design Time plugin for Eclipse can only be used on systems that have IE installed – and therefore not on Linux!
Special Features
Sandor discussed and demonstrated a number of especially interesting tools and features. Among these the XSD (Schema) Mapper in JDeveloper, the Adapter Framework for bindings with Services implemented using many different technologies and the Workflow Service that replaces the User Tasks.
Transformer tasks and XSD Mapper
As stated previously, all variables in BPEL Process are XML Documents. Assigning variables with values or manipulating the variables can be done with so-called Copy Rules. These copy rules act on a single variable and are defined with XPATH expressions. Oracle BPEL provides Transformer Tasks – Assign with XSLT – that use an XSLT stylesheet to transform an XML Document and thus manipulate potentially many variables.
The JDeveloper 10.1.3 provides a very cool tool: the XSD Mapper. In this tool, you load two XML Schema documents (XSDs). One is the source and the other obviously the target. The XSD Mapper tool is used to derive an XSLT stylesheet that can transform XML Documents conforming to the source XSD into XML Documents conforming to the target XSD.
The tool has you draw connecting lines between source and target elements. You can include aggregators, transforming functions, splitters etc. From this design, the XSD Mapper tool creates an XSLT document. It provides a tester where you can either generate an XML Document conforming to the source XSD or load such a document, then perform the transformation based on the generated XSLT and verify whether the resulting document conforms to the target XSD – and whether the document lives up to your own expectations. The final XSLT can be used in the Transform tasks in BPEL processes – and of course in your own applications!
Inspired by this tool I was wondering whether there is an – preferably open source – tool that helps me construct a first-cut XSD from an XML document. Sandor and Andrej joined me in an enthousiasm for the idea as well as a brief Google search, with no immediate results – apart from some commercial offerings. If anyone has a suggestion – please let me know! By the way, Sandor indicated that Oracle is contemplating making this XSD Mapper tool publicly available – as shareware/freeware, possibly as Eclipse plugin. That would be a nice gesture!
Adapter Service Wizard
In my view, BPEL will most often be used within organizations to implement processes that span across applications and technologies. In such circumstances it is usually not desirable to invoke all services through the WebService SOAP based protocols, because of the overhead involved. Perhaps the most useful feature of the Oracle BPEL Design Time, especially the current version of the JDeveloper plugin, is the binding framework and the Adapter Service Wizard.
Oracle BPEL Process Manager provides a binding framework that supports invocation of services implemented in various technologies through a number of adapters. The JDeveloper plugin for BPEL Design Time includes a wizard that will configure the bindings for various services.
The wizard supports already an interesting list of technologies – with also some notable omissions: Java and EJB. Supported are a File Adapter, an FTP Adapter, an adapter for AQ (Advanced Queuing), a Database Adapter, a JMS adapter and an Oracle Workflow adapter.
The Database Adapater supports pretty direct calls to PL/SQL Stored Procedures! Database tables can also send and receive messages. The database adapter uses JDBC calls to the database and is internally based on Oracle’s Toplink framework. This adapter is an alternative – much directer and easier to use – for writing your own Java wrapper that invokes the PL/SQL procedure and calling that Java ‘stub’ as Service from the BPEL process. AQ provides another, indirect way of invoking Database functionality, possibly wrapped itself in JMS.
Another interesting case is the File Adapter. A File Service can act as Receptor of calls but also as Sender. In that last case, the BPEL Process would poll the existence of the file to be bound into the process and only proceed once that file were available. The Binding also supports conversion of the file – which can be in XML Document format but also in comma-separated-value or fixed positions or even free format – into a (XML) BPEL variable. The Adapter Service Wizard for the File Adapter type includes a tool that allows mapping of the File Format to an XML format or vice versa. That allows processing of files as BPEL variables. Oracle BPEL has proposed a new standard for describing the format of non-XML files: N-XSD (with N for native I believe). This file format mapping to XML tool looks remarkably similar to the File Source to Table Target Mapping Wizard in Oracle Warehouse Builder; has there been close cooperation between different development teams in Oracle?
Sandor agreed that a simple Java binding was a remarkable omission from the list of Adapters supported in the wizard. He explained it through the eyes of the BPEL Development Team that argued that supporting Java binding would also have to include EJB binding which in turn means that the adapter, wizard and generated configuration became much more complex. Unfortunately there has not yet been enough time to deal with all that complexity so no Java binding support is available at all. It is the intention however to provide this support in the production release (10.1.3) . How to invoke Java based services from BPEL is described in many samples and documents, for example Anthony Reynolds on using Java from BPEL .
Workflow Wizard and Workflow Engine
Many processes at some stage require some form of user interaction. To approve, act in an exception(al) situation, decide in less-than-clearcut circumstances, process non-structured data etc. Previously, Oracle BPEL worked with User Tasks – asynchroneous Services to be invoked by the BPEL engine that had to completely be implemented by the developer. With the new release, Oracle BPEL has the concept of workflows. Workflows are scopes – almost independent sections – within BPEL processes that always contain decision points where the result of a user action is tested and a next step is chosen depending on that result. The actual structure of the Workflow – a sub-process within the overall BPEL process in which through user participation a certain state is reached – depends on the specific business process of course. However, there are many recurring patterns in such workflows and the JDeveloper BPEL plugin has a Workflow Wizard that caters for a number of such patterns – and even allows you to add more patterns that you can reuse across BPEL processes.
In the wizard the developer provides values for the configuration parameters of the selected workflow pattern – such as the names of the various escalation levels , the decisions that can be made at every stage and the associated variables. Once deployed, the BPEL Server provides a User Interface – the Worklist Express – that allows users to retrieve their worklists and participate in the processes that require their attention.
It is pretty easy to develop custom tailor-made User Interfaces to support the workflows, but this generic front-end looks pretty usable as well.
Resources
Andrej Koelewijn’s Blog on this session: Oracle BPEL jDeveloper plugin
Mike Lehmanns blog: Lehmann Blogger – Web services, J2EE and occasionally more …
IBM Article on WebServices Gateway: An introduction to Web Services Gateway – A proxy for Web services
Anthony Reynolds’ Blog on WebServices and BPEL; for example his post on using Java from BPEL where he discusses three possible approaches ( * Calling the code directly using the bpel :exec construct.
* Wrapping the Java code as a SOAP Web Service.
* Wrapping the Java in WSIF.
) and describes in detail the WSIF based approach
Orchestration Standards Move Ahead By Greg DeMichillie posted: May 19, 2003 with background and history for BPEL.
BPEL and J2EE: Bringing the Best of Both Together by Mike Lehmann (Thanks to J2EE 1.4, there is now a natural partnership between the J2EE developer building Web services and the BPEL developer fashioning these services into business processes) How can the BPEL developer take advantage J2EE applications? How can the J2EE developer take advantage of BPEL processes?
BPELJ: BPEL for Java technology – A joint whitepaper by BEA and IBM Level: Advanced March 22, 2004
An Introduction to Service-Oriented Architecture from a Java Developer Perspective by Debu Panda, 26th January 2005, appeared in OnJava. Discusses SOA, Java & WSDL and JAVA-RPC.
No Responses