Today I attended – and presented at – the Database Systems 2006 conference – an annual Dutch conference on Database Architecture and Technology, Enterprise Integration and Service Oriented Architectures as well as Information Architecture and Business Intelligence. The main theme – at least in the sessions I attended – was SOA. In this article a brief overview of the main highlights of the day. By the way: the best highlights by the way most definitely included my conversations with Sandor (Oracle), Ivan (Logica CMG) and Frank (Whitehorses).
SOA and Data Quality
The first session I sat in on was by Ivan Pellegrin, Oracle and especially Business Intelligence at Logica CMG. Ivan is one of the big promotors of Oracle technology both within his own company and in the market place as a whole. He has co-organized an impressive string of sessions (Guru4Pro) with technical specialists and visionaries. His presentation consisted of two parts: one on SOA and the other on Data Quality. Ivan listed the top 19 thoughts with regard to SOA.
His top 5: 5. SOA helps achieve business agility 4. Building Blocks 3. SOA can be an evolution – do not consider it a revolution 2. It is still maturing 1. It is achievable today.
In a brief quiz, Ivan stressed the fact that having an application that invokes a couple of web services does not imply you are leveraging a SOA (this realization earned me a drinking mug – thank again Ivan). The second quiz question asked whether SOA makes distributed computing easier. The conclusion was that initially it is quite the opposite: it will make Distributed Computing initially harder.
Ivan made the audience think about loose coupling. It is not just that at a technical level the components are decoupled because they are defined through and invoked through there interfaces. It is also the decoupling between the business level – the process definition as formulated by the business – and the technical level, the actual implementation of that process. When we have implemented our business process using various services, we can easily modify that process without having to change the implementation of the services. At the same time, we can change the service implementation – for example leveraging a new third party system we have acquired – without impact on the business, as long as the service’s interface is unchanged. There is even a decoupling of sorts between business processes.
Ivan is a very charming presenter, who knows how to engage the audience, breaks the tension and potential boredom with a quiz and a lot of nice visuals in his Powerpoint Presentation (perhaps the technology I learned most about today is actually Powerpoint).
The second part of the presentation, on Data Quality, was not all that much related to the first part. In the end, the concept of Data Hubs and central data services gave this section a SOA twist as well, but it was essentially about the quality of data per se. Ivan stressed how there are not global standards for assessing the quality of data. Terms such as accuracy, correctness and completeness spring to mind. Or Fitness for Use. Ivan very much rejected this last qualification: if you can predict all uses your business and end users will make of data for all time to come, you can start making a living as a medium! There is never such a thing as a generally agreed upon ‘fitness for use’.
Yet Data Quality is a very important issue – especially if we want to exchange the data as part of service invocations or if we want to make data services available. Data Quality issues can have serious impact on business operations. From misunderstandings to manual conflict resolutions and unhappy customers. Or simply failure for services to interact. Ivan illustrated the absurdity in many cases: "The management gets involved and states it wants to have some data quality." (After 45 minutes, the term XML was mentioned only for the first time. )
Ivan discussed how as part of ETL processes where Data Warehouses are loaded from operational databases organisations often implement data cleaning and enrichment. Typically Data Warehouses will contain the improved version of the data. Now we could wonder: why are these data quality improvements not fed back to the operational data sources? An interesting question – one that we did not go in furthermore.
What we did discuss next are the Data Hubs – centralized Single Source of Truth data stores and collections of services. Oracle for example provides hubs for Customer-data, Financial Consolidation and Product Data. These hubs are capable of synchronizing the data they contain to all localized collections of data. This should only be necessary until all our applications can deal with the data services provided by the data hub instead of locally stored data (and until the performance of such services will we adequate).
Web 2.0
Peter Hinssen did a very entertaining presentation on Web 2.0. Not too much substance, no technical content but certainly amusing and with some useful tidbits. Web 1.0 – Netscape, based on the old business model for software (like the horseless carriage, the first incarnation of the car that was modeled after horse-drawn carriages), with a front end focus and B2C interaction – vs. Web 2.0 – represented by Google, service based with a focus on Cross Channel Consistency and end-to-end integration, focus on B2B integration. See for example the recent announcement of Google acquiring Writely, an on-line word processor (http://news.com.com/Google+buys+Web+word-processing+technology/2100-1032_3-6048136.html). Also see Alfresco (http://dev.alfresco.com/, an open source Content Management System from a team of former Interwoven/Documentum developers, recently published. Another example of the new Web 2.0 business model: CODAF – open source ERP.
Web Service Security and other musings
Guy Crets, a Belgian consultant, did a more technical presentation on his experiences with Web Services and B2B integration. He gave an overview of various standards or attempts at standards for specific message types. He wondered why – when for EDIFACT message types have been defined and confirmed under United Nations (UNCTAD) responsibility and a wide range of message types have been confirmed and is actively used – no real effort at standardization for XML messages has been attempted by anyone. He briefly discussed sub-standards, like ebXML – waning and on the verge of collapse -, RosettaNet – successful but only for a niche of the market – and others. But he also stated that this lack of general standardizations brings many organizations to defining their own message ‘standards’. This works against B2B integration, especially beyond the bilateral level.
Guy said in his opinion there is far too much focus on Synchronous WebServices while in many situations an asynchronous service is far more appropriate. One of the issues with the asynchronous service is the fact that calling party needs to be called itself by the service when that service has a reply ready. However, the caller may well be behind a firewall, out of reach for the service. Guy described how the WS-Polling standard describes a mechanism whereby the calling party will make additional polling calls to the service to inquire whether an answer to the service request is already available. Another solution could be the use of an intermediary – at the edge of the firewall for example – that will receive the reply from the WebService and somehow make it available to the calling party. Guy mentioned several marketplayers that offer hub-like services to B2B partners, much like the VAN providers in the EDI domain. He specifically mentioned ViaCore, E2Open en HubSpan. Grand Central is another party that offers services to facilitatie WebService based integration.
Guy also wondered why there has not been set up a standard for SOAP communication via Email. Given the omnipresence and maturity of the SMTP protocol, using emails for communicating SOAP messages seems like a very obvious thing to do. For EDI traffic, apparently in addition to Http and Ftp, Email based traffic is very common, said Guy.
He concluded with a number of recommendations and statements: – if you are going to get yourself an ESB, make sure that it is at least based on JMS (Java Message Queue). JMS is so widely supported, it can almost be considered the de facto standard for MOM (message oriented middle ware). – when defining a (Web)Service, start with specifying the Schema Definition (XSD) – when developing a (Web)Service, start by developing the WSDL (WebService Definiton Language) document, only then start writing code that implements that contract. This sounds like Interface Driven Development, which is very close to Test Driven Develoment or Specification Driven development – for testing WSDL based web services, Guy recommended the Open Source tool soapUI (see http://www.soapui.org/: soapui is a desktop application for inspecting, invoking and testing (functional and load) of web services over HTTP. It is mainly aimed at developers/testers providing and/or consuming web services (java, .net, etc). Functional and Load-Testing can be done both interactively in soapui or within a automated build/integration process using the soapui command-line tools. soapui currently requires java 1.5 and is licensed under the LGPL license). He made some glowing comments about this product.
Finally, Guy recommended a number of books: 
: Web Services Platform Architecture : SOAP, WSDL, WS-Policy, WS-Addressing, WS-BPEL, WS-Reliable Messaging, and More (Paperback) by Sanjiva Weerawarana, Francisco Curbera, Frank Leymann, Tony Storey, Donald F. Ferguson (Prentice Hall March 2005) ISBN: 0131488740 :Understanding SOA with Web Services (Independent Technology Guides) by Eric Newcomer, Greg Lomow – Addison-Wesley Professional (December 2004), ISBN: 0321180860
Starting out with SOA – small, simple and very tangible
Our own presentation on SOA was obviously one of the highlights of the conference. You can download the presentation – largely in Dutch – here: SOA op jouw werkvloer. We have a good crowd – 35 – for one of several parallel vendor sessions. I believe that we got the message across quite well: SOA – you can start small, the first project can already be profitable and IT can flexibly support the business (maybe even lead the business) rather than be the bottleneck. All in all: the IT Department can become a business accelerator.
