XL Deploy: Simple Case of Custom Deployment

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

The last couple of months I have been assigned to a project dedicated to deployment automation including several levels and types of technologies: database, middleware, services and portals. The selected deployment tool is XL Deploy from XebiaLabs.

One of the main challenges was to determine the proper order of the steps within the deployment plan considering the dependencies between the several artifacts.

Continue reading

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Use the inbound REST adapter of StreamExplorer to consume events from HTTP POST requests

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

StreamExplorer is a fairly recent product from Oracle – a business user friendly layer around Oracle Event Processor. In various previous articles, I have discussed StreamExplorer. I have demonstrated how SX can consume events from JMS, EDN and from CSV files. This article shows how a stream in StreamExplorer can expose a REST service to which events can be pushed.

image

In this case, we will look at a movie theater. More specifically, we will monitor the visitors entering and leaving the various rooms in the theater so we keep track of the number of people currently present in each room. When rooms are almost full, we may have to stop selling tickets or perhaps open up an additional room showing the same movie. When a room is [almost]empty, perhaps we should cancel the show altogether. When people keep coming and going, there may be a problem we should attend to.

In this case, the events are received by Stream Explorer in the form of JSON messages posted to a REST service. We use a SoapUI test case with requests to send in these events. The test request called aprtyOf3InRoom1, isshown in the figure:

image

The request is configured to be sent to the endpoint http://localhost:9002. That is the endpoint for Stream Explorer (and OEP and more specifically the Jetty server running the OEP domain on top of which Stream Explorer was applied). The (REST) Resource is specified as /cinema. Together this means that this request is sent as a POST request to the end point http://localhost:9002/cinema. So that is where our StreamExplorer application will have to consume the message.

The message itself has a JSON payload with two simple properties: room (to identify a room in the movie theater) and partySize (to stipulate the number of people involved in an observation). Note: the number of people is positive when a party enters the room and negative when it leaves the room.

The TestSuite TestSuiteCinema contains a single test case with a number of steps that simulate events on a slow night in the movie theater.

Create a Stream and a First Exploration for Handling Cinema Events

Create a new Stream. The name is not crucial. The Source Type should be REST.

image

 

Specify the Context Path as /cinema.

image

Define the REST Shape. Set the name to CinemaEntryOrExitEvent. Define two properties: room – of type String – and partySize – of type Integer.

image

Click on Create to create the Stream.

The Exploration wizard opens next. Set a name for the exploration – for example RoomOccupancy. You may provide a description as well.

image

Click on Create.

Configure the Exploration for example like this:

image

The events are aggregated grouped by room. The aggregation to be calculated is a sum over the partySize. This should produce the total number of people in every room. In this case, I have also chosen to have the summary calculated once every 5 seconds and to include in the results only the events from the last 8 hours (which is quite arbitrary).

At this point I create a test case in SoapUI for the REST service – and run it:

image

The exploration starts reporting its findings:

image

 

Because publishing data to a REST service in JSON format is so easy – and is supported from many tools such as SoapUI and technologies including PL/SQL and Java, it is a very convenient way of sending events to a StreamExplorer application, both for development and testing purposes as well as for a production implementation.

Using the PL/SQL procedure described in this article  we can easily send events from inside the Oracle Database to the StreamExplorer stream:

image

– and verify the effects in Stream Explorer. Note that the movie showing in room 2 must be quite awful;-)

image

 

Resources

Getting Started with the REST Adapter in OEP 12c – A-Team Blog

Make HTTP Post call from PL/SQL

Make HTTP Post call from Java

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Unleash the power of Java API’s on your WLST scripts!

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Oracle SOA Suite and many other Oracle products have extensive Java API’s to expose their functionality. WLST can often be used for relatively course grained actions. WLST (the version supplied in Weblogic 12.1.3) uses Jython 2.2.1. Jython is the Python scripting language implemented on the Java Virtual Machine. Jython allows easy integration with Java. In this article I describe how you can unleash the power of these Java API’s on your WLST scripts!
WLST on the Java road

Continue reading

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Demonstration of Oracle Stream Explorer for live device monitoring – collect, filter, aggregate, pattern match, enrich and publish

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

This article describes a use case for Oracle Stream Explorer – Oracle’s business user friendly interface on top of OEP – Oracle Event Processor. We assume a large number of devices – such as printers, copiers, sensors, detectors, coffee machines – spread across the globe – and the cloud.

image

All devices continuously report their status, by sending a message every other second that contains their device identifier, a code that can indicate the healthy status or an error and some additional details. The sheer number of devices combined with the continuous stream of reports they sent in set the challenges perimeters within which we have to implement fast and effective monitoring. Our specific challenge is: “whenever a device reports an error code three times within 10 seconds, we consider that device broken, and action should be taken” (that also means that we do not spring into action on the first or even second fault report from a device). Additionally: we only require a single action for a broken device – once the action is initiated, we do not have to start an action again for that same device – unless of course it is broken again at a much later point in time.

image

The concrete implementation described in this article looks as follows:

image

For the sake of a simple demonstration, we read device message reports from a csv file, instead of a live stream such as a JMS destination or an HTTP channel. Note that the Stream Explorer implementation would be exactly the same for these other stream types. Stream Explorer processes the device signals. For signals that satisfy the requirements of a broken device, the information is enriched from a database with device details – such as the physical location of the device – and finally an EDN event is composed and published. This event is consumed by a SOA Composite application in the SOA Suite 12c environment. This composite can virtually do anything, including assigning a task, starting a BPM process or sending an email.

The implementation described in this article is also demonstrated in a video on YouTube:

Demo of Stream Explorer in action for monitoring devices

Implementing the Stream Explorer application

With Stream Explorer – everything starts from a Stream – a source of events or messages such as a JMS Queue or Topic, the SOA Suite Event Delivery Network, an HTTP channel or a CSV file such as in this case. Through one or more Explorations – that each can do filtering, aggregation, enrichment and pattern matching – finally conclusions can be published to a target. Targets can be JMS destinations, HTTP channels, a CSV file and the Event Delivery Network of SOA Suite.

In a number of steps, we will go from the CSV file with device signals to the EDN events that represent broken devices.

image

The first step will be an exploration that filters the non-ok signals from the stream:

image

The second step will find failing devices by counting the number of non-ok signals in a 10 second period and filtering on any device with a count greater than or equal to 3:

image

Next, to prevent any failing device from being reported more than once (in a certain period of time) we perform deduplication, using one of the special patterns shipped out of the box in Stream Explorer:

image

The remaining messages report a unique failing device and we need to enrich those messages with details about the device location, taken from a Reference defined for a database table:

image

The enriched messages are routed to a target: an EDN event that is published to the SOA Suite runtime whose address is configured:

image

The next sections show – just as the video does – how these explorations are created in Stream Explorer.

Open Stream Explorer and log in

image

image

Open the Catalog.

DeviceSignals Stream and NonOkSignals Exploration

Create a new Stream, of type CSV.

image

Press Next. Select the CSV file:

SNAGHTML14b4e240

You may want to briefly inspect this file:

image

Back in the wizard, refine the Data Shape definition and give it a name:

image

Press Create to complete the Stream definition.

Next, the Exploration wizard is started. Set the name of the Exploration to NonOkSignals:

image

Press the Create button.

The Exploration editor is showing and the first messages read from the CSV file are produced in the exploration:
image

Define a filter for the exploration, to only produce messages for non ok error codes:

image

Publish the exploration and return to the catalog.

Exploration for Failing Devices

Create a new Exploration. Call it FaultyDevice[s]and use NonOkSignals as the source.

image

Press Create to navigate to the editor for the exploration.

Specify a summary: count DeviceId , group by DeviceId. Next, add a filter, to produce results only when for a DeviceId we have counted 3 or more NonOkSignals. Finally, specify a time window over which to calculate the count. Set the range of the window to 20 seconds and set the evaluation frequency to [once every]3 [seconds].

image

Publish the exploration. Note that if you wait for some time, you will see devices being reported in the Live Output Stream section of this page:

image

Return to the Catalog.

Deduplicate Devices

Create a Pattern – an exploration based on a predefined pattern. Pick the Eliminate Duplicates pattern.

image

Select FaultyDevice as the source for this pattern exploration. Select DeviceId as the key to determine the uniqueness by (eliminate the second and subsequent events from the FaultyDevice stream). Set the window to 1 minute. This means that any subsequent events for a device are blocked during 1 minute after the initial event for the device. After that minute, the slate is cleared for that device.

 

Publish this exploration too.image

Return to the catalog.

Enrich the FailingDeviceEvents and Pulish them as EDN Event

Create a Reference:

image

Note: before doing this, a Data Source to the schema that contains the database table should have been set up in OEP – either through the WLEvent Visualizer or directly in the OEP configuration file:

image

The table used as a reference is shown here:

image

Define the Reference – set its name and its type:

image

Then select the table that provides the data for this reference:

image

And press Create.

Now create a new exploration – our last one in this article:

image

Set its name and select Exploration4 – the name auto-assigned to the [Eliminate Duplicates] Pattern based exploration – as the source:

image

Add the Reference DeviceDetails as a second source for this exploration:

image

Then specify the correlation condition:

image

This completes the logic for the Exploration. We do need to add a target to it – to make the outcomes from this Exploration produce EDN events.

Click on Configure Target. The select EDN as the target type in the wizard:

image

Type in the connection details for the SOA Suite managed server. The EDN event definitions will be loaded.

image

Note: at this stage, there seems to be an issue when the EDL files import XSD definitions that themselves import XSD definitions. Additionally, it seems that EDN events with nested elements are not handled well by Stream Explorer.

In this case, my SOA Suite domain contains just a single EDN event definition that satisfies the conditions mentioned above. Select this definition:

image

Next, provide the mapping between properties in the current exploration and the elements in the EDN event:

image

And press Finish.

Publish the Exploration to make it active.

After some time, new failing devices will have be spotted and EDN events will have been published. These EDN events in this have trigger the SOA Composite DeviceMaintenance. Here we see an example of some instances as well as the details of one of these instances.

image

The device signals picked by Stream Explorer and processed through four explorations result in this SOA composite being invoked – only once – for every device that has found to be failing. The hard in terms of live observing a great number of devices simultaneously and continuously is taken on by Stream Explorer. And configuring these explorations turns out to be quite straightforward and rather declarative.

Resources

Zip with CSV file with device data and SQL scripts for the creation of the device details table: deviceMonitoringResources .

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack – Part 2

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

This article continues the story from Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack – Part 1. It is the story of how to expose SOAP/XML based Web Services – primed for enterprise integration and process automation – as REST services with URL parameters and JSON payloads that provide the mobile backend APIs required for the development of mobile apps. The story began with the challenge Luc Bors and I were facing for the Live Mobile Hacking session at the Oracle Fusion Middleware EMEA Partner Forum (March 2015). For an audience of some 200 SOA Suite and Oracle Middleware experts – including the product management teams for several of the products we were going to demonstrate – we were to develop live both a mobile app as well as a mobile backend (REST APIs) to support the mobile app.

The use case was defined – an app for flight attendants to be used on tables to look up flight details, inspect the passenger list, update the flight status and record passenger complaints. Luc handled the mobile app based on the agreements we made on the REST resources, services and payloads. It was up to me to implement the backend – consisting of a database, a SOA composite to expose the database data and operations at middleware level (in SOAP/XML) – as described in the previous article. Next, I needed to implement a Service Bus project to expose the required REST service interface with JSON payload, leveraging the SOAP/XML service published by the SCA Composite. That is the story told in this sequel – the part highlighted in the red rectangle in the next picture:

image

This article will briefly describe the steps I went through to create the REST services on top of the SOAP/XML services. At the end of this article, the end to end flow from the above illustration will have been implemented.

All source code for the mobile backend can be downloaded from a GitHub repository.

 

Steps:

  • Design REST services, Resources, URLs and JSON payloads
  • Create REST proxy services in Service Bus project
  • Create Pipeline to connect REST proxy to Business Service for SOAP/XML services with appropriate transformations between message formats (JSON/XML and vice versa)
  • Publish Service Project and test end-to-end REST services (all the way down to the database and back again)

At the end of this article, here is what we have achieved:

image

 

At this point, the Mobile App can come off the mock services it has undoubtedly used during development, and start consuming the real services.

 

Step One – Design REST services, Resources, URLs and JSON payloads

Luc and I had discussed the functionality to be offered in the mobile app. Subsequently, Luc indicated the REST services he would require to implement the app. Both in terms of functions and in terms of the actual URLs patterns and JSON payloads. The latter can be found in this GitHub directory.

The former were defined as follows:

To retrieve flight details: GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07 (with a JSON response structured as shown below)

image

 

To retrieve the passenger list for a flight : GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>/passengerlist?flightDate=2015-03-07 (with a JSON response structured as shown below)

image

 

 

To update the status of a flight : PUT request to /FlyingHigh/FlightService/FlightService/flights – with a JSON request payload as follows:

image

 

The REST service for submitting a complaint from a passenger was defined the other way round: I dictated the service interface and Luc simply had to consume it – as an example of how ideally you should not do things in mobile app development. The call for submitting the complaint has to be sent as a POST request to: soa-infra/resources/default/CustomerService!1.0/CustomerCareService/complaints  with a JSON payload structured like this:

image

Note: the CustomerCareService is exposed from a SOA Composite directly, not through the Service Bus proxy that is created in the remainder of this article. The flow for requests to this service is as follows – a BPM process being triggered as a result:

image

 

Step Two – Create REST proxy services in Service Bus project

The REST adapter wizard in JDeveloper can be used in two ways: either take an existing Pipeline (Service Bus) or Component (such as Mediator or BPEL) and derive a REST binding from the existing SOAP/XML service interface (not an ideal approach) or design a REST service with corresponding payload as desired and have the adapter wizard generate the internal SOAP/XML representation that can be connected through a Pipeline or Mediator. The latter is the approach to be used when the REST service design is leading – as it usually is, including this case.

The starting point is a Service Bus project that already contains a Business Service for the FlightService SOA Composite. This business service obviously exposes the same SOAP/XML interface exposed by the composite.

image

What we are after, is a REST service exposed by this Service Bus project – of which the interface definition is predefined  by Luc. We can create this from the context menu in the Proxy Services swim

lane. The wizard for configuring the REST binding appears.

SNAGHTML2333454

Set the name to FlightRestService.

Now for each operation the service should expose, a Resource should be configured. For example for the retrieval of FlightDetails using /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07

 

Press the gears icon to specify the format for the response payload. This is expressed through an XSD document that describes the native format for the response; we call this type of document an NXSD document – in this case for example nxsd_retrieveFlightDetails.xsd.

Choose the type of format:

SNAGHTML24b19c2

 

And load the sample of the JSON message payload:

SNAGHTML24becba

The wizard will now create the NXSD representation of this JSON message format:

SNAGHTML24d59bf

Close the wizard.

Back in the Response tab, we can generate a sample of the message payload in JSON or XML format:

 

image

The operation to fetch the passengerlist can be defined a similar way:

SNAGHTML2515c6d

and the details:

image

 

Note: the endpoint definition for the REST service is set like this – a bit silly really, with the duplicate FlightService that we will have to use when invoking the service, for example from the browser:

image

 

Step Three – Create Pipeline to connect REST proxy to Business Service

Create Pipeline to connect REST proxy to Business Service for SOAP/XML services with appropriate transformations between message formats (JSON/XML and vice versa).

Add a Pipeline (for example FlightPipeline) that implements the WSDL created for the REST Proxy Service. The REST proxy can invoke this pipeline, handing it the XML representation (described by the NXSD document) of the URI parameters received in the REST call. It expects to receive the XML representation (also described in the NXSD) of the JSON payload of the response message.

XQuery of XSLT based transformations are required between the NXSD description of the request and the XSD description of the input to the business service FlightServiceBS and between the output from the FlightServiceBS and the required NXSD format to be returned to the REST Proxy Service.

The next figure lists the six transformations created between NXSD and XSD-for-Business Service:

image

 

The pipeline between REST proxy and Business Service contains an operational branch – for the three REST operations in the FlightService. In each operation, it does similar things: transform the request, route to the business service and transform the response.

image

The overall project looks like this:

 

image

 

Step Four – Publish Service Project and test end-to-end REST services

Nothing special about deployment, so I will skip that part.

After deployment, the REST services can be invoked. An example is:

image

This results in a SOA composite instance with the following flow trace:

image

We can look at the details of what happens inside the Mediator:

image

to learn that what we get in JSON is pretty much what was constructed in the PL/SQL package in the database. No surprise there.

Of course I can invoke the other REST services as well:

image

but that does not add much value.

The really interesting next step is when Luc starts implementing the mobile app against these REST services.

SNAGHTML21929f7

 

Perhaps the Passenger Complaint is mildly interesting – as it causes some activities, as illustrated by the next two pictures:

first the call to the REST service for (POSTing) customer complaints:

image

and then the result inside the SOA Suite and BPM Suite:

SNAGHTML21b9c87

 

 

Some artist’s impressions from the actual live demo

 

Here some pictures from the actual demo, courtesy of Luis Weir:

image

image

image

image

and Sten Vesterli:

image

and Jim Weaver:

image

and Vikas Anand:

image

and Jan van Hoef:

image

and José Rodrigues

image

and Simon Haslam:

image

and Jacco Cijsouw:

image

 

and one last one taken by me from stage:

image

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page