Quick Introduction to Oracle Stream Explorer – Business User friendly processing of real time events

The new Oracle Stream Explorer provides us with a [business] user friendly facility to process real time events. Through visually appealing and functionally intuitive web wizards, Stream Explorer has us construct explorations that consume events from streams, process these events through filtering, aggregation, pattern matching and enriching and deliver these events to downstream destinations.

Using Stream Explorer, we can tap into streams of events – frequently JMS messages and alternatively HTTP PUB/SUB, SOA Suite EDN events or REST calls. For testing and demo purposes, we can use an CSV file as the source for a stream exploration. A stream is fed into one or more explorations that do the interpretation and processing of the events. A target can be associated with an exploration to have the outcomes of the exploration – which are also events, at a more elevated level after all the processing has taken place – delivered for subsequent action or communication. Destination types available for targets are JMS, REST Service, HTTP Pub (channel) and the EDN of SOA Suite. Again, for development, testing and demonstrations, a CSV file can be set as the target.

In this first introduction to Stream Explorer, we will discuss a very simple challenge: we are organizing a small conference. In three rooms, sessions take place simultaneously. Our attendees are free to decide which session to attend. We would like to know at virtually any moment how many people are in each room. We have set up simple detectors at the doors of the rooms that produce a signal whenever someone enters or leaves the room. This signal consists of the room identifier (1,2 or 3) and an IN/OUT flag (values +1 or -1).

We will use StreamExplorer to process these events and produce an aggregate per room of the net number of people that entered or left the room. In subsequent articles we will do more advanced explorations in this same setting – looking at prematurely concluded sessions, jammed doors, overflowing rooms etc.

Preparation

The preparation consists of the installation of Stream Explorer on top of an OEP domain. Start the OEP Server. The Stream Explorer can be accessed at http://host:port/sx. Login using the same user used for the OEP Events Visualizer web application: wlevs.

The CSV file with room events is created as follows:

You can see people entering into the rooms. Row 15 has the first attendee leaving a room – room 2 – apparently not too happy with the session taking place.

And Action…

Login to Stream Explorer:

On the welcome page, go to catalog:

Create a new Item of type Stream:

Specify the name and a description for the Stream. Set the type – in this case CSV file:

Click on Next. Select the CSV file to upload – specify the interval (delay between reading events from the CSV file) and the initial delay:

Click Next.

Define the shape of the event data object – in this case with two attributes RoomId and InOutFlag.

Click on Create to complete the definition of the (input) Stream and have it created.

The wizard to define the exploration on top of this streams opens subsequently. Specify name , description and tags (optionally).

Click on Create.

Configure the Exploration with a summary (sum InOrOutFlag group by RoomId) and (first click on the clock icon) a time window (some fairly large value) and an Evaluation Frequency of 10 seconds – meaning that a summary is published every 10 seconds – if during that 10 seconds the value changed for the room.

The Live Output Stream in the bottom of the page starts showing events:

The results from the exploration can be forwarded to an external destination. Click on Configure a Target. Specify a CSV file as the target:

The exploration is published when the definition is complete.

From this moment on, the OEP application created on the background is live – deployed to the OEP server and active.

The results can be seen in the CSV file that is being appended to:

Here we see that after some 40 seconds,  11 people have gathered in room 3. Room 2 around that time has just a single attendee – after having had 5 at its peak. Note that finding maximum occupancy would be an easy task using Stream Explorer: just add another exploration that processes the outcome from the exploration created in this article – and have it aggregate with the MAX operator.

In the visualizer, we can check in on the OEP applications that were created in this example:

We can also export the Explorations from Stream Explorer and refine them as OEP applications in JDeveloper.

For details, documentation and download, see the OTN Pages.