On the integrity of data in Java applications – presentation from JFall 2013

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

The accuracy, internal quality, and reliability of data is frequently referred to using the term ‘data integrity’. Without it, data is less valuable or even useless. This session takes a close look at what data integrity entails and how it can be enforced in multi-tier application architectures using distributed data sources and global transactions. The discussion will make clear which elements are required from any robust implementation of data oriented business rules aka data constraints and it will explain how most existing solutions are not as watertight as is frequently assumed. Steps for achieving reliable constraint enforcement are demonstrated.

The presentation I did last week for the JFall 2013 conference can be checked on SlideShare:

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

OOW13: First glimpses of the new SOA Suite 12c

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

During this week’s Oracle OpenWorld Conference, we were given some sneak peeks into the short term future of the Oracle SOA Suite. During various roadmap sessions, on the demo grounds as well as in the keynote session by Thomas Kurian (the replay of which you can see here: http://medianetwork.oracle.com/video/player/2687033686001), new features were described and demonstrated, allowing us to get a fairly good overview of what is going to come for SOA Suite – later in 2013 and sometime in 2014 (probably the first half of that year).image

The SOA Suite plays an important part in the three themes Thomas Kurian set down for the Fusion Middleware suite of products: support for mobility, cloud and business user empowerment.

Some of the highlighted new aspects of Oracle SOA Suite are:

  • Adapters to connect from on-premise to in-the-cloud – specifically targeting SalesForce, RightNow and also providing an SDK to create custom integrations into the cloud (the first cloud adapters will be released on 11g, before the end of the year)
  • Mobile enablement by exposing RESTful services that communicate using JSON as well as adding the capability to call out to such services (12c functionality)
  • Enhanced functionality on Exalogic (of course it runs faster on Exalogic, up to 20 times)
  • Modular runtime with a lighter footprint.

A brief demonstration of the Cloud Adapter was given by Demed L’Her during said keynote. The next screenshot shows the Adapter wizard for the Cloud Adapter.

image

image

It allows the developer to pick a specific operation for a specific business object exposed by RightNow (or SalesForce) (the adapter knows about the APIs exposed by RightNow and SalesForce):

Continue reading

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

On the Integrity of Data

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

To be useful, data held and used in information systems has to live up to a number of expectations. The data should be an accurate representation of its source. It should be reliable. The data should have internal consistency. The data should adhere to rules based on the logic of the real world. This accuracy, internal quality, and reliability of data is frequently referred as data integrity.

Safeguarding the integrity of data is a challenge, one that increases in complexity when multiple users access and manipulate the data simultaneously , obviously a common situation. And that challenge reaches new heights when the data is managed in multiple independent data stores rather than a single database.

Figure 12

Earlier this month, the Oracle Technology Network published an article that I recently wrote on this subject: http://www.oracle.com/technetwork/articles/soa/jellema-data-integrity-1932181.html. I was triggered into writing it by two recent experiences.

One was at a customer of mine where we are designing a service oriented architecture, based on a number of distinct and independent data domains. These domains are exposed through elementary (entity) services. A second tier of composite (or business) services exposes functionality that may involve multiple data domains. We have had and are still having discussion about how to implement data integrity constraints and how to manage transactions that span across data domains. In order to ensure we all had the same understanding of what exactly the challenges are, I decided to record my understanding of integrity, constraints and transactions.

The second situation was at a different customer. There they stated that data quality and absolute robustness of the enterprise database was essential. And they went on to explain how they had implemented their integrity enforcement using Java based logic. Their implementation was elaborate and impressive – but not robust. They would enforce attribute and record level constraints just fine, but any constraint involving multiple records was not enforced with the rigor they needed and thought they had achieved. They had forgotten to properly take into account the multi-user/multi-session environment in which their logic would be used (as well as the other entry point into the database that completely by-passed their business logic). Here again I was compelled into writing down what enforcing integrity entails, specifically the need for locking in a multi-session environment.

I hope the article that was triggered by these two cases – and many cases before that – will help other organizations and teams as well, in understanding what data integrity and enforcing constraints in a truly robust way entails. Frequently, they will find that the current implementation is not in fact robust. For example many organizations using PL/SQL and Trigger based ‘business rule enforcement’ have not implemented a proper locking mechanism and are therefore not as well protected against data corruption as they typically think they are.

Continue reading

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

How Oracle Database uses internal locks to make statement level constraint validation robust at the transaction level

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Data Constraints are essential in protecting the integrity of the data in any relational database. The Oracle Database provides four types of declarative constraints that help implement various types of data rules. These are: Primary Key, Unique Key, Foreign Key and Check Constraint. Although these constraints can be configured to be enforced at transaction time (when the transaction is committed) by setting them to be deferred, the default behavior of the constraints is to enforce the integrity rule at statement level. That means that when a transaction performs multiple DML statements, the constraints are validated during execution of the statement. When the statement is done, the integrity is ensured (otherwise the statement would have failed) and additional statements can be executed.

In the multi session environment that is the Oracle Database, you could be wondering – as was I – how the Oracle Database ensures that other sessions executing DML operations can not undermine the integrity of the data touched by the current transaction. And how it can do so without needlessly preventing other sessions to perform data manipulation using various forms of locking.

It turns out that the database use a smart, fine grained way of locking to prevent sessions from corrupting the integrity established by statement level validation. I will explain what it does.

Let’s take for example table EMP. You know, the one with the 14 employees that have ENAMEs, EMPNOs and other columns. By default, no constraints are defined on table EMP. Let’s add a unique constraint on ENAME:

image

Let’s now assume we have two concurrent database sessions, X and Y.

In session X, the EMP record for SCOTT is updated. SCOTTs ENAME is set to SCOTTY. This statement succeeds, because SCOTTY does not already exist in the ENAME column. Session X does not commit the transaction [yet]. In the concurrent session Y, the name change for SCOTT cannot yet be seen as it has not yet been committed. In this session Y, another record is updated: MILLER is updated to become SCOTT.

image

At this point, the question is:

  • is this update statement accepted by the database (because by now SCOTT is already SCOTTY because of the update in session X)
  • will it fail because in the context of Session Y there already is a SCOTT (because the transaction in Session X is not committed and the update of SCOTT therefore is not visible in Session Y)
  • will table EMP be locked for any updates in session Y because session X is already updating it?
  • will something else happen?

Try for a moment to answer this question yourself. What do you think will happen?

Before I give you the correct answer, I will introduce a second situation: In session X, a new employee is inserted. The statement succeeds as the constraint validation finds no fault with this new record. The transaction in session X is not yet committed. In session Y, the SMITH record is updated, to WILSON, a value that does not currently exist in table EMP.

image

The question again is: will the update statement in Session Y succeed? Fail? Be blocked?

In this case: the update in Session Y will succeed as it is completely unrelated to the insert in session X. Different records are involved, different – non conflicting values for ENAME are used. There is no problem.

The next figure shows how this second example continues. Session Y goes on to insert a new Employee record with ENAME WILMA. This statement too succeeds. Then session Y attempts to rename Employee KING; the ENAME value for KING is updated to FRED – which is also the ENAME inserted for the new Employee from Session X.

image

Session X did not yet commit. The new FRED record is not yet visible in session Y. On the other hand, the insert succeeded. If the database would also accept this update in session Y too because no FRED currently exists in table EMP, then after both transactions have been committed we would end up with two FREDs – in violation of the Unique Key constraint.

So what do you think will happen in session Y with the update statement:

  • is this update statement accepted by the database (because the new FRED from session X is invisible in session Y so there is no reason to conclude a uniqueness violation)
  • will it fail because clearly we cannot end up with two FRED records and since the insert in session X is already accepted there is no alternative but to reject this update
  • will table EMP be locked for any updates in session Y because session X is already manipulating it? (well clearly this cannot be the case because in session Y the update of SMITH was accepted without any problem
  • will something else happen?

Again, try for a moment to answer this question.

As it turns out, the update statement in  session Y will be blocked. It runs into a lock held by session X. This lock is not held on a record, nor is it held on the table. It is a lock on the Unique Key constraint on ENAME in combination with the value FRED. It is a fine grained lock that does not prevent other sessions from performing insert and update statements that set ENAME to other values than FRED. But using the value FRED is prevented by this lock. A lock that session X will hang onto until the transaction either commits or rollback.

Once the lock is released after the commit in Session X, the update statement in session Y can continue. And it will fail because the validation of the unique key constraint runs into the FRED employee record that is now committed.

image

The former case is now simple to understand: the update of MILLER in session Y runs into a similar lock as in the second example with KING’s update to FRED. The lock is held by session X. When session X commits, the lock is released, session Y’s update can continue and the row update will be succesful because at that time, SCOTT has formally been renamed to SCOTTY.

image

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

Fanning Out Events on the Oracle SOA Suite 11g Event Delivery Network

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page

On the Oracle Technology Network, architecture section, my article titled “Fanning Out Events on the Oracle SOA Suite 11g Event Delivery Network” has just gone live:

image

This article describes:

how events can be used in Oracle SOA Suite 11g to have business processes impact each other in a meaningful way with maximum decoupling. Specifically, this article describes a solution for fanning out Event Delivery Network (EDN) events to a more fine-grained level. This allows a single event to influence multiple running instances of a Business Process Execution Language (BPEL) process. The article uses the following Oracle SOA Suite 11g components: BPEL , Mediator ,Event Delivery Network, Spring , Locator API , Composite Sensors

Share this on .. Tweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Email this to someoneShare on TumblrBuffer this page