Weblog for the AMIS Technology corner
Posts tagged correlation
Batch Aggregation of files in BPEL process instances based on correlation
Feb 24th
Remco is an interesting guy with unexpected ideas springing from a creative brain. He can make life interesting, challenging and puzzling. This time he had another interesting challenge – not all that weird to be honest. The challenge in short was:
Our invoicing system produces files that contain one or more invoice entries. Every entry describes an invoice for a certain company we do business with. There can be multiple invoice entries for the same company. The objective is to aggregate together all invoice entries for a company – potentially from many different file. For each invoice entry – some special processing involving service calls is required. Once all entries for a company have been collected and aggregated, some additional action is required – for example recording the company invoice aggregate in a database or in a file and call a webservice to perform additional processing. The files with invoices are produced over a period of a couple of hours. It is important that the processes performing the aggregation are reliable – they should not lose any entries.
The specific question we investigated is: can we solve this puzzle using Oracle SOA Suite 11g? And an early approach towards applying the SOA Suite’s capabilities to this challenge was based on BPEL’s correlation mechanism. In short: every company for which the batches contain invoice entries wil have an instance of a composite called InvoiceAggregator. This instance is carried by a BPEL component that has correlation configured on CompanyId. In our test set up, we have the company instance expire after 5 minutes: it will cease aggregation when it has not received new messages for a period of 5 minutes.
Composite InvoiceProcessor contains a File Adapter that reads the Invoice entries from files arriving in a specified directory. Each entry is passed to a Mediator that forwards it to a BPEL component. This BPEL component instantiates the company specific instance of InvoiceAggregator (if it does not already exist). Then it passes the invoice entry to that instance.
Composite InvoiceProcessor is stateless: after processing an individual invoice entry, it terminates (typically in a couple of 100ms). Composite CompanyAggregator is around for much longer – in our set up at least for 5 minutes and typically longer when multiple invoice entries arrive for the company.
Note: this lay out is a simplification of the real challenge. The essence of the correlation based interaction is captured in this example however.
SOA Suite 11g – Composite instance as WebService Result Cache using BPEL Correlation
Sep 17th
This article describes an architectural pattern, implemented in the Oracle SOA Suite 11g, that is somewhat similar to the Oracle Database 11g Function Result Cache. It introduces a SOA Composite Application in the role of Result Cache. In its most simple form, the result cache is initialized – loaded with values -, used by other SOA applications that need the cached values, refreshed/reset when required and terminated. Through this ‘result cache’ – frequently used and not-so-frequently changed values that are published by (possibly remote, expensive or slow-reacting) web services or adapter services can be made available to local consumers in a simple, cheap and fast manner. We achieve this using the fast native SCA binding used for invoking in-container services exposed by fellow composite applications (that’s for speed) and the BPEL correlation mechanism (that’s for finding the result cache in the first place).
What the actual value of this pattern and implementation are is not yet entirely clear to me. Your feedback is appreciated. What I do know is that this article is also a good demonstration of using correlation and of applying some of the XML manipulation techniques available in BPEL processes.
In a single picture, here is what I created: one or more Composite Instances are initialized; each instance is a cache, identified through its name. Upon initialization, the cache is loaded with values – supposedly important, frequently reused, hard-to-get-by values. Composite applications that are interested in these cached values can retrieve them from the cache, by calling the exposed service and passing the name of the cache (here is where correlation kicks in) and passing the name of the cache entry. The BPEL process looks for the entry and returns the value when found or null and an appropriate message when not found.
