This pattern was ‘invented’ by our SOA-architects team integrating the Oracle SOA Suite and the Oracle e-Business Suite in an existing IT-landscape.
The trickle-feed integration pattern can be used when
distributed systems integrate with one another by means of asynchronous messaging.
The messages are versioned to each other in such a way that the newest version may
only be processed by the receiving system after all older versions of
that message are successfully processed. The trickle-feed mechanism is
responsible for the correct processing order of messaging between all message
versions. The mechanism keeps track of the already send versions and will send
out only the correct version to the distributed system on a trickle-feed basis,
while keeping other versions on a stack until they are ready to be processed
Context
Your environment is a service oriented system or a
distributed and possibly heterogeneous system with independent cooperating
components. The messaging mechanism is asynchronous reply-response messaging.
Problem
The main problem lays in the fact that decision for sending
a newer version is based on the response. Because the response is received
asynchronously, this means of messaging poses some implementation difficulties.
Suppose you have a system that is sending messages, for
example an application sending out purchase orders and related order lines to (a)
distributed system(s). For each purchase order, changes in order and order
lines are reflected in sending a new version of the purchase order.
The purchase order is processed by the distributed system
and a response is send back asynchronously. The response message holds
information stating if the message was processed successfully or has failed. Only
when the response states success, a new version, when available, can be
processed to the distributed system.
Other purchase orders, which have no correlation to the
above purchase order, are to be processed without any restriction. Table 1
schematically shows an example of the sequenced events.
Table 1: sequenced
events
| Event time | User action | Purchase order list | Reply | Response |
| T1 | Create PO A, version 0 |
PO-A0 | No other versions of A exists à send PO to distributed system |
Processing… |
| T2 | Change PO A |
PO-A1 | Response for version 0 is not received à no not send |
|
| T3 | Create PO B, version 0 | PO-B0 | No other versions of B exists à send PO to distributed system |
Processing… |
| T4 | ß response: PO-A0 is successfully processed |
|||
| Decide next version in sequence of PO-A to send à send PO-A1 |
Processing… | |||
| T5 | ß response: PO-B0 is successfully processed |
|||
| Decide next version in sequence of PO-B to send à no more messages |
Solution
A solution must provide a decision mechanism between send messages
and correlated responses and decide which the next message to send is.
The solution is to forward each message to the “trickle-feed
handler†as shown in figure 1. This handler is responsible of the following
tasks:
–
Store/persist all messages.
–
Whenever a message arrives, decide if the
message is of the latest version. If so, send the message to the distributed
system, using asynchronous messaging. Otherwise, take no action to send it.
–
Handle the response messages. The messages must
be correlated. If response states a processing failure, either do nothing.
If the response states success, decide which the next version in sequence to
send is. Delete the successfully processed messages from the message store.
The trickle-feed handler’s responsibility is correct
handling of messages; therefore successfully processed messages are deleted
from the store. History keeping is beyond the responsibility of the handler.
Figure 1: Trickle-feed
handler and activities
In situations where the request-response cycle is shorter
than the creation of new versioned messages, the trickle-feed has only little
overhead in respect to ‘normal’ asynchronous request-reply, because messages
are stored, deleted and a decision is made before the message is send. However,
when the creation of new versions is faster than the request-response cycle or
when the target system is down, messages are stagnated at the handler until the
target system back up again or has processed the previous messages successfully.
This is where the trickle-feed pattern blossoms.
Dependencies
This pattern is depending on the Correlation Identifier pattern (Hophe, G., Woolf, B., 2004).
References
Hophe,
G., Woolf, B. (2004). Enterprise Integration
Patterns : Designing, Building, and Deploying Messaging Solutions, Addison Wesley.
Buschmann,
F., Meunier, R., Rohnert, H., Sommerlad, P.,Stal, M. (1996). Pattern-Oriented Software Architecture: A System
of Patterns, Addison Wesley.
