Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack – Part 2

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

This article continues the story from Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack – Part 1. It is the story of how to expose SOAP/XML based Web Services – primed for enterprise integration and process automation – as REST services with URL parameters and JSON payloads that provide the mobile backend APIs required for the development of mobile apps. The story began with the challenge Luc Bors and I were facing for the Live Mobile Hacking session at the Oracle Fusion Middleware EMEA Partner Forum (March 2015). For an audience of some 200 SOA Suite and Oracle Middleware experts – including the product management teams for several of the products we were going to demonstrate – we were to develop live both a mobile app as well as a mobile backend (REST APIs) to support the mobile app.

The use case was defined – an app for flight attendants to be used on tables to look up flight details, inspect the passenger list, update the flight status and record passenger complaints. Luc handled the mobile app based on the agreements we made on the REST resources, services and payloads. It was up to me to implement the backend – consisting of a database, a SOA composite to expose the database data and operations at middleware level (in SOAP/XML) – as described in the previous article. Next, I needed to implement a Service Bus project to expose the required REST service interface with JSON payload, leveraging the SOAP/XML service published by the SCA Composite. That is the story told in this sequel – the part highlighted in the red rectangle in the next picture:

image

This article will briefly describe the steps I went through to create the REST services on top of the SOAP/XML services. At the end of this article, the end to end flow from the above illustration will have been implemented.

All source code for the mobile backend can be downloaded from a GitHub repository.

 

Steps:

  • Design REST services, Resources, URLs and JSON payloads
  • Create REST proxy services in Service Bus project
  • Create Pipeline to connect REST proxy to Business Service for SOAP/XML services with appropriate transformations between message formats (JSON/XML and vice versa)
  • Publish Service Project and test end-to-end REST services (all the way down to the database and back again)

At the end of this article, here is what we have achieved:

image

 

At this point, the Mobile App can come off the mock services it has undoubtedly used during development, and start consuming the real services.

 

Step One – Design REST services, Resources, URLs and JSON payloads

Luc and I had discussed the functionality to be offered in the mobile app. Subsequently, Luc indicated the REST services he would require to implement the app. Both in terms of functions and in terms of the actual URLs patterns and JSON payloads. The latter can be found in this GitHub directory.

The former were defined as follows:

To retrieve flight details: GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07 (with a JSON response structured as shown below)

image

 

To retrieve the passenger list for a flight : GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>/passengerlist?flightDate=2015-03-07 (with a JSON response structured as shown below)

image

 

 

To update the status of a flight : PUT request to /FlyingHigh/FlightService/FlightService/flights – with a JSON request payload as follows:

image

 

The REST service for submitting a complaint from a passenger was defined the other way round: I dictated the service interface and Luc simply had to consume it – as an example of how ideally you should not do things in mobile app development. The call for submitting the complaint has to be sent as a POST request to: soa-infra/resources/default/CustomerService!1.0/CustomerCareService/complaints  with a JSON payload structured like this:

image

Note: the CustomerCareService is exposed from a SOA Composite directly, not through the Service Bus proxy that is created in the remainder of this article. The flow for requests to this service is as follows – a BPM process being triggered as a result:

image

 

Step Two – Create REST proxy services in Service Bus project

The REST adapter wizard in JDeveloper can be used in two ways: either take an existing Pipeline (Service Bus) or Component (such as Mediator or BPEL) and derive a REST binding from the existing SOAP/XML service interface (not an ideal approach) or design a REST service with corresponding payload as desired and have the adapter wizard generate the internal SOAP/XML representation that can be connected through a Pipeline or Mediator. The latter is the approach to be used when the REST service design is leading – as it usually is, including this case.

The starting point is a Service Bus project that already contains a Business Service for the FlightService SOA Composite. This business service obviously exposes the same SOAP/XML interface exposed by the composite.

image

What we are after, is a REST service exposed by this Service Bus project – of which the interface definition is predefined  by Luc. We can create this from the context menu in the Proxy Services swim

lane. The wizard for configuring the REST binding appears.

SNAGHTML2333454

Set the name to FlightRestService.

Now for each operation the service should expose, a Resource should be configured. For example for the retrieval of FlightDetails using /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07

 

Press the gears icon to specify the format for the response payload. This is expressed through an XSD document that describes the native format for the response; we call this type of document an NXSD document – in this case for example nxsd_retrieveFlightDetails.xsd.

Choose the type of format:

SNAGHTML24b19c2

 

And load the sample of the JSON message payload:

SNAGHTML24becba

The wizard will now create the NXSD representation of this JSON message format:

SNAGHTML24d59bf

Close the wizard.

Back in the Response tab, we can generate a sample of the message payload in JSON or XML format:

 

image

The operation to fetch the passengerlist can be defined a similar way:

SNAGHTML2515c6d

and the details:

image

 

Note: the endpoint definition for the REST service is set like this – a bit silly really, with the duplicate FlightService that we will have to use when invoking the service, for example from the browser:

image

 

Step Three – Create Pipeline to connect REST proxy to Business Service

Create Pipeline to connect REST proxy to Business Service for SOAP/XML services with appropriate transformations between message formats (JSON/XML and vice versa).

Add a Pipeline (for example FlightPipeline) that implements the WSDL created for the REST Proxy Service. The REST proxy can invoke this pipeline, handing it the XML representation (described by the NXSD document) of the URI parameters received in the REST call. It expects to receive the XML representation (also described in the NXSD) of the JSON payload of the response message.

XQuery of XSLT based transformations are required between the NXSD description of the request and the XSD description of the input to the business service FlightServiceBS and between the output from the FlightServiceBS and the required NXSD format to be returned to the REST Proxy Service.

The next figure lists the six transformations created between NXSD and XSD-for-Business Service:

image

 

The pipeline between REST proxy and Business Service contains an operational branch – for the three REST operations in the FlightService. In each operation, it does similar things: transform the request, route to the business service and transform the response.

image

The overall project looks like this:

 

image

 

Step Four – Publish Service Project and test end-to-end REST services

Nothing special about deployment, so I will skip that part.

After deployment, the REST services can be invoked. An example is:

image

This results in a SOA composite instance with the following flow trace:

image

We can look at the details of what happens inside the Mediator:

image

to learn that what we get in JSON is pretty much what was constructed in the PL/SQL package in the database. No surprise there.

Of course I can invoke the other REST services as well:

image

but that does not add much value.

The really interesting next step is when Luc starts implementing the mobile app against these REST services.

SNAGHTML21929f7

 

Perhaps the Passenger Complaint is mildly interesting – as it causes some activities, as illustrated by the next two pictures:

first the call to the REST service for (POSTing) customer complaints:

image

and then the result inside the SOA Suite and BPM Suite:

SNAGHTML21b9c87

 

 

Some artist’s impressions from the actual live demo

 

Here some pictures from the actual demo, courtesy of Luis Weir:

image

image

image

image

and Sten Vesterli:

image

and Jim Weaver:

image

and Vikas Anand:

image

and Jan van Hoef:

image

and José Rodrigues

image

and Simon Haslam:

image

and Jacco Cijsouw:

image

 

and one last one taken by me from stage:

image

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

In a previous article – https://technology.amis.nl/2015/03/03/preparation-for-live-mobile-hacking-with-an-ofm-12c-red-stack-budapest-2015/– I introduced the challenge Luc Bors and I were facing for the Live Mobile Hacking session at the Oracle Fusion Middleware EMEA Partner Forum. For an audience of some 200 SOA Suite and Oracle Middleware experts – including the product management teams for several of the products we were going to demonstrate – we were to develop live both a mobile app as well as a mobile backend (REST APIs) to support the mobile app.

The use case was defined – an app for flight attendants to be used on tables to look up flight details, inspect the passenger list, update the flight status and record passenger complaints. Luc handled the mobile app based on the agreements we made on the REST resources, services and payloads. It was up to me to implement the backend – consisting of a database, a SOA composite to expose the database data and operations at middleware level (in SOAP/XML) and a Service Bus project to expose the required REST service interface with JSON payload, leveraging the SOAP/XML service published by the SCA Composite.

This article will briefly describe the steps I went through. All source code for the mobile backend can be downloaded from a GitHub repository.

image

 

Step One – Database

The database created for Flying High contains just five tables – as shown in the next figure.

image

However, these tables are encapsulated – hidden from view from external consumers. This I consider to be a best practice. For various reasons that include the encapsulation of (potentially complex) SQL statements in APIs in the database rather than in client applications and the freedom of changing the actual implementation of tables and the way data is stored. The API that is externally exposed is a PL/SQL package FLIGHT_SERVICE. The package specification contains three unit – two functions and a procedure – for retrieving flight details, the passenger list for a flight and for updating the status of a flight.

The input and output parameters for these units are all based on user defined types. This allows me to exchange nested data structures in single round trips, provide data structures that map very well to XML messages and that can very well be handled by the database adapter. Additionally, use of these UDTs allow the database developers to work in a rather elegant way – leveraging their tools to their full potential.

 

image

 

When you check the script database.sql in the source code you will find the definitions of the tables, the UDTs and the PL/SQL package.

 

Step Two – SOA Composite, Database Adapter, SOAP/XML Service

Each of the operation in the database package Flight_Service is exposed as Reference in a SOA Composite application using three database adapter bindings. The User Defined Type based inputs and outputs to these operations are represented in (very similar) XML structures in the interface definitions for these three bindings.

 

image

 

In parallel with the creation of the PL/SQL package, a WSDL was created for the FlightService, with a similar granularity in operations and message data structures. Note: this is not required, but it certainly makes life simpler when you have to do a pressure cooker demo. The WSDL is visualized here:

image

You can access the source itself through this link. The FlightService.wsdl has a companion XSD document that describes the request and response messages as we want to expose them to consumer of the SOAP/XML service.

The structures in the XML Schema Definitions are quite close to the User Defined Types in the PL/SQL API. However, they are defined from a canonical model – not from database table and column definitions.

image

A Mediator component was added to the

For each of six back-and-forth hand overs, I have  created XSLT maps to convert from canonical input message to the input required by one of the three database adapter bindings as well as to convert from each of three outputs from the database adapters bindings to the response message format specified in the FlightService WSDL and XSD.

image

The Mediator has three routing rules configured that each use two XSLT Maps. The maps are quite straightforward, thanks to the similarities between the PL/SQL API object structures and the XSD definitions. One example of an XSLT map is shown here:

image

This map uses a DVM (domain value map) lookup to convert between the code used for FlightStatus in the database (sch, brd,cld) and the canonical values to be used in the response message (scheduled, boarding, closed). The DVM definition shown next. The document was transferred to the database MDS under the runtime SOA Suite.

image

 

After deploying the SOA composite (and configuring both the Data Source and the Database Adapter connection), the FlightService can be invoked, for example from SoapUI:

image

This screenshot shows a call to the getFlightDetails operation and the resulting response – wrapped in a SOAP envelope and of course all XML in nature.

Each service call will start a new instance of the FlightService SOA Composite. The instance will make the call to the database. All interaction can be tracked – with a  fine grained enough audit level at least – as shown in the  next figure.

image

This figure shows the XML structure that came out of the Database Adapter binding – and that was converted from the User Defined Type at database level. Note the value of BRD for FLIGHT_STATUS that is converted by the DVM lookup to boarding.

Unfortunately this beautiful service is still not good enough for Luc and his Mobile App. A REST/JSON service was the agreement. So my next challenge: expose the FlightService as a REST service that uses JSON payloads. That story is told in a follow up article.

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Oracle Service Bus: Obtaining a list of exposed SOAP HTTP endpoints

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

The Oracle Service Bus is often used for service virtualization. Endpoints are exposed on the Service Bus which proxy other services. Using such an abstraction layer can provide benefits such as (among many other things) monitoring/logging, dealing with different versions of services, throttling/error handling and result caching. In this blog I will provide a small (Java) script, which works for SOA Suite 11g and 12c, which determines exposed endpoints on the Service Bus.Exposed SOAP HTTP endpoints

Continue reading

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Resolving deployment issues with Service Bus 12c – OSB-398016 – Error loading WSDL

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

I was completely stuck with Service Bus 12c project deployment from JDeveloper to the Service Bus run time. Every deployment met with the same fate: Conflicts found during publish – OSB-398016, Error loading the WSDL from the repository:  The WSDL is not semantically valid: Failed to read wsdl file from url due to — java.net.MalformedURLException: Unknown protocol: servicebus.

I was completely lost and frustrated – not even a simple hello_world could make it to the server.

SNAGHTMLc3d51e6

Then, Google and Daniel Dias from Link Consulting to the rescue: http://middlewarebylink.wordpress.com/2014/07/17/soa-12c-end-to-end-e2e-tutorial-error-deploying-validatepayment/. He had run into the same problem – and he had a fix for it! Extremely hard to find if you ask me, but fairly easy to apply.

It turns out this is a known bug (18856204). The bug description refers to BPM and SB being installed in the same domain.

The resolution:

Open the Administration Console for the WebLogic Domain. From the Services node, select service OSGi Frameworks:

image

Click on the bac-svnserver-osgi-framework link. Note: if you run in production mode, you will now first have to create an edit session.

Add felix.service.urlhandlers=false in the Init Properties field for the configuration of this service. Then press the Save button.

image

If you run in Production Mode, you now have to commit the edit session.

Then, in order have this modification make any difference, you have to restart the WebLogic (Admin) Server.

This resolved the issue for me – a weight was lifted of my shoulders. Thanks to Daniel from Link!

Edit (28th December 2014): See for more details and a patch from Oracle this article by Jan van Zoggel: http://jvzoggel.wordpress.com/2014/10/07/patching-the-oracle-service-bus-12-1-3-unknown-protocol-deployment-error/

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

SOA Suite 12c: Weekend Roundup

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Almost 100 hours ago Oracle SOA Suite 12c was released and 72 hours ago BPM Suite 12c had it turn. At AMIS we held a 50+ hours blog-a-thon and published a total of 25 blogs. This blogs looks back at the release of SOA Suite and BPM Suite 12c.

On Thursday 26th of June at 09:00 AM PST (06:00 PM CET) Oracle release SOA Suite 12c. Right after the release we published the first blog with the most striking features. Together with us a lot of people followed what happened next on twitter, the 12c madness exploded. Right after the released the first people downloaded the software and posted the first pictures and experiences with the installation of the SOA Suite quick start. The whole night I followed the tweets from the SOA community and the product managers of Oracle that made it all happen.

Below a selection of related posts we published in the last few days

First look at SCA Composite features This blog will summaries the features specific to SCA Composites / BPEL and the Enterprise Manager Dashboards. The features are summarised, but most will get an own blog that tells about the full details. The following features are discussed; Changed project / directory structure, Project / Component en Scope Templates, updates to the composite editor, updates to the mediator, updates to the BPEL component / activities, the Fault Policy Editor,SOA Composer refresh in 12c and the SOA Suite Debugger. Read more.

All about Developer Productivity and Performance  This blog will summaries the features specific to the Developer Productivity and Integration in JDeveloper and in the Enterprise Manager. The features are summarised, but most will get an own blog that tells about the full details. The following features are discussed; Developer installer, Integration of SOA and SB development, New technology adapters, New Database Connection Types and Performance & Optimization. Read more.

First look at Service Bus features This blog will summaries the features specific to Service Bus and the Enterprise Manager Dashboards. The features are summarised, but most will get an own blog that tells about the full details. Oracle renamed the product from OSB (Oracle Service Bus) to SB (Service Bus), apparently it also stands for Oracle Secure Backup. The following features are discussed; Integrated with JDeveloper, Splitting Proxy from Pipeline, Kick-start an Service Bus (SB) project using templates, MDS support for importing shared resources and possibility to enable/disable activities. Read more.

The Star of the SOA Suite 12c – The Pipeline Every new product release has a star. And of course picking that star is arbitrary. One person’s favorite feature may not be someone else’s. My pick of the 12c crop is … (drum roll) …. the Pipeline! The Pipeline is new – to some extent. In Service Bus 12c, instead of a Proxy Service that contains a Message Flow as we used to have in previous incarnations of OSB and Aqualogic Service Bus, we now have two separate components: the Proxy Service and the Pipeline (which contains the Message Flow). Read more.  

Introducing the REST Adapter. Calling a RESTful service returning JSON The REST Adapter is an important new feature of Oracle SOA Suite 12c. This new adapter allows easy calling/exposing of RESTful services (see https://technology.amis.nl/2014/05/14/what-is-rest/ for an introduction to REST) based on a WADL or configured manually. Not only does Oracle introduce the REST Adapter, but JSON support is also added in SOA Suite 12c. These features will greatly increase the integration options and flexibility of Oracle SOA Suite. In this blog I’ll describe how a RESTful service can be called by using the REST adapter. Read more.

BPEL Inline Sub Processes or locally Reusable Chunks of BPEL process logic An in-line sub process can best be thought of as a callable scope. It is a scope that is not part of the normal process flow, but rather a unit of encapsulated logic that can be called zero, one or multiple times from anywhere within the BPEL process – even recursively from within itself. It is in effect very similar to a private member function in a Java Class or a procedure in the body of a PL/SQL package. Read more.

Sharing resources between development environments: Export from and Import to Design Time SOA MDS repository  The MDS repository is an important mechanism for SOA Suite 12c, both at design time during development as well as at run time. At design time, MDS is used to host shared resources such as WSDL and XSD documents that are referenced in many different places. Instead of copying these resources between applications and projects,  MDS is concept that is native to JDeveloper and therefore much easier to use with these shared resources. Read more.

Dynamically overriding attributes of static routing rules in Mediator component Routing Rules in a Mediator component carry several attributes that govern the routing rule run time behavior. These include: XSLT or XQuery used for transformation, the filter expression, target operation, the parallel or sequential indicator, the Schematron document used for validation, the native validation and the value assignment (for example for header properties). Read more.  

Where to find Service Bus Pipeline Alerts In SOA Suite 12c, the Service Bus console no longer exists. The Service Bus specific run time browser interface is not intended for administration activities: all administration around SOA Suite 12c is consolidated into Enterprise Manager Fusion Middleware Control. So a valid question now becomes: how to locate pipeline alerts produced in 12c Service Bus run time environments? These are of course to be found in EM FMW Control. But where and how exactly? Read more.

Native Format Translation in Service Bus Pipeline – XML to and from JSON, CSV and fixed position formats Data offered as internet resource in a structured XML or non-XML format (such as JSON, CSV or fixed position) – cannot be retrieved by the File Adapter, but it can be read by the Service Bus. Using a combination of an HTTP transport style Business Service, a Service Callout and a Native XSD mapping, it turns out to be fairly easy to implement what is best described as a remote file adapter. Read more.

Using Domain Value Map (DVM) in Service Bus projects Domain Value Maps are still DVMs in SOA Suite 12c. Create them in JDeveloper, use them in Transformations and Assignments in Mediator and BPEL. Inspect them and edit them at run time using SOA Composer. Nothing really new there. Read more.

BPM Suite 12c: Oracle Business Rules – Verbal Rules In this blog post I will transform a Decision Table step-by-step into the new Verbal Rules. As a starting point I created a composite with a BPM, a BPEL and a Rules component. Both BPM and BPEL components use the same Rules component. See the following three screenshots. You can download the sample application with this starting point from here. Read more.

Neat little feature in Service Bus Pipelines: Disable/Enable Activity to ease development Sometimes it is just a small feature that can make life easier. SOA Suite 12c is full of large, sweeping new components and grant improvements. And it also has a ton of smaller things. One such smaller new thing is the ability in Pipelines in Service Bus projects to disable an activity. Sounds perhaps like something you would typically not want to do – add an activity then disable it. Read more.

Support for reusable XQuery Libraries | Modules It featured in many forum discussions (such as this one) in the last few years: “The Oracle Service Bus XQuery engine fully supports all of the language features that are described in the World Wide Web (W3C) specification for XQuery with one exception: modules”. See for example the W3C spec. Library Modules would make it possible to create reusable XQuery functions, that are available for use in any XQuery transformation. Read more.

First steps with the Coherence Adapter to create cross instance state memory One of the new adapters shipped with SOA Suite 12c is the Coherence Adapter. This JCA adapter makes it easy for a Service Bus business service or a SOA composite application to interact with a Coherence memory grid. Fully declaratively and with very little trouble, data can be put on a Coherence grid (aka cache) and read from that cache. The cache is accessed like a big map: using a key, an object is saved to and retrieved from the cache. The cache is accessible across service executions and process instances, as well as across cluster nodes. Read more.

Property window for inspecting and editing BPEL activities Before release 12c, editing activities in a BPEL process was done in the wizard that could be activated by double clicking the activity or by selecting edit from the context menu on an activity. Alternatively, a developer could resort to the BPEL process source file to make changes to the definition of activities. In 12c, the property inspector has been extended to also support editing the properties of BPEL activities. When the property inspector window is available, it will synchronize with the currently selected activity in the BPEL editor. Read more.

BPM Suite 12c: Quick Start installation – 20 minutes and good to go Oracle released BPM Suite 12c. Just like SOA Suite 12c – released the day before – this release comes with the quick start option: to quickly start going through development and test iterations, the development environment (JDeveloper + BPM Studio) is now equipped with an Integrated WLS that contains the BPM Suite 12c run time engine. All one needs to not only develop but also to run a BPM process is packaged in a single environment that is installed from a single file. Read more.

For full list check our SOA Suite 12c overview.

Mentionable Blogs from the community

SOA Suite 12c is available for download SOA Suite 12c is a major step forward to become an “Industrial SOA” and developer productive solution. Key new features include: Performance & Optimization (Modularity, Composite Lazy Loading, WLS Work Manager Based Tuning) and Diagnosability & Manageability (Task Based Enterprise Manager, Enterprise Scheduler Service).  Read More.

Using the clone ability to duplicate a Service Bus 12c project (by Jan van Zoggel) It’s quite common in a service oriented landscape that a newer version of a service is required. For instance due to new functionality for 1 service consumer which brakes the contract for the other consumers. JDeveloper 12c has a cool feature helping us to clone a Service Bus project. Read More.

Getting started with Oracle Service Bus 12c: importing 11g sources (by Laurens van der Starre) One of the new features is that Oracle Service Bus development is integrated into Oracle jDeveloper Studio. One way to get started quickly is by simply importing your 11g service bus sources into 12c. First export your 11g sources, either from the Servicebus Console or Eclipse (OEPE)… Then, this sbconfig.jar is easily imported into the new Oracle jDeveloper Studio 12c (as expected). Read more.

Also twitter exploded, below are my favorite tweets

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page