Comments on: Introducing Materialized Views as mechanism for Business Rule implementation (complex declarative constraints) https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/ Friends of Oracle and Java Sat, 01 Aug 2015 11:10:05 +0000 hourly 1 http://wordpress.org/?v=4.2.3 By: AMIS Technology blog » Blog Archive » On the false sense of security with PL/SQL based implementation of business rules - and what to do about it https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1949 Sun, 22 Oct 2006 16:53:54 +0000 /?p=475#comment-1949 […] Building applications in our corner of the world typically revolves around databases. The most important bit about an application is not the application, but rather the data it creates and processes that gets stored in the database. And even before security and performance, the integrity of that data is our prime concern. Making absolutely sure our database is robust is essential. Enforcing all data oriented business rules is a sine qua non for a successful application.In this article I will discuss a typical way of implementing business rules in the Oracle database, using PL/SQL and table triggers. An approach taken by thousands of organisations and ten of thousands or more developers. An approach proven over the years, one would think. An approach used in a business rule frameworks like CDM RuleFrame. And an approach that is fundamentally flawed. Date centric business rules come in a number of varities. Some pertain to attributes – like NAME is a String of up to 25 characters or SALARY may not exceed 10,000 or JOB can be one of the following values. Others apply to tuples -entire records – like: an employee with JOB unequal to SALESMAN may not have a (value for) COMMISSION, a CLERK may not earn more than 3000 or You cannot Manage yourself (EMPNO should not be equal to MGR). These rules can be implemented using declarative constraints in the database – CHECK CONSTRAINTS. Other declarative constraints are Primary Kyy and Unique Constraints and Foreign Key or Referential Constraints. The former allow us to implement rules like no two employees may have the same name and job (that would be utterly confusing) or there can be only one employee in each job per department. The latter enforces rules such as every Employee must belong to a Department. Implementing a business rule using a declarative constraint is perfect. These constraints are built into the database kernel, are optimized for performance and are guaranteed to be enforced. When these constraints are enabled, we have the certaintity that the data in our tables complies with the rules. That is as robust as it can get.However, more complex data rules cannot be enforced using these declarative constraints. Let’s take a look at a simple example. Again in the EMP and DEPT tables in the SCOTT schema – you may have seen them before – we have the following rule: "no more than three clerks in a department" .  Other options: Unique Expression Based Index  and Check Constraints on Materialized ViewsUsing Unique Index on Expressions – see for example Using unique function based index as check constraint – we can, still declaratively, enforce rules like: no more than one CLERK per Department or not more than one birthdays per department per month. But even then at some point declarative solutions fall short, and we have to turn to PL/SQL for implementing advanced complex rules.Another option is available through Materialized Views. Though perhaps not best known for the role they can play in implementing and enforcing business logic, Materialized Views nevertheless are a powerful instrument for rules that are too complex for normal declarative constraints or Exression Based Unique Indexs. The MV route is clean: no triggers or supporting packages are needed. Note that the Materialized View option – see also Introducing Materialized Views as mechanism for Business Rule implementation (complex declarative constraints) – powerful as it may be is still limited to a subset of complex business rule we will have to implement in our databases.Applying the Materialized View approach to our example of ‘no more than three Clerks per Department’ we could implement that rule as follows (for explanation and details see the article referred to above):Create a Materialized View Log for Table EMP: since we want the rule to be enforced at commit time, we need a REFRESH ON COMMIT Materialized View and for that we need to have Materialized View Log; note that this View Log can be reused for all complex business rules concerning table EMP.Create a Materialized View on table EMP for this particular Business Rule; we may be able to reuse the MV for other business rules as wellCreate a Check Constraint on the Materialized View that enforced the business ruleThe steps are pretty simple, especially compared to what we have seen before when implementing the same rule using PL/SQL:1. Create the Materialized View Log for table EMPcreate materialized view log on emp with rowid(deptno, job) including new values; 2. Create the Materialized View for the Business Rulecreate materialized view emp_br001_mvrefresh fast on commitasselect deptno, count(*) cntfrom empwhere job = ‘CLERK’ group by deptno/ 3. Add the Check Constraint to enforce the rulealter table emp_br001_mvadd constraint check_cntcheck ( cnt < 4)deferrable; Note: since the MV will only be refresh when the transaction commits, the check constraint will take effect only at that time. It is somewhat like a deferred (declarative) constraint. In the actual time-line, this constraint will be enforced after all ‘normal’, table based deferred declarative constraint were successfully enforced. That is a good thing as you would like to enforce this business rule at the end of the transaction, rather than after each individual statement. Transferring Clerks between department would otherwise become hopelessly complex.Putting this MV based implementation to the testThe starting point is the following:SQL> select ename 2 , deptno 3 from emp 4 where job = ‘CLERK 5 order 6 by deptno 7 /ENAME DEPTNO———- ———-MILLER 10SMITH 20ADAMS 20JAMES 30 Now we open two database sessions and make some changes in the data: […]

]]>
By: AMIS Technology blog » Blog Archive » The Hunt for the Oracle Database On Commit trigger https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1948 Mon, 07 Aug 2006 07:44:42 +0000 /?p=475#comment-1948 […] Back in 1998 or 1999 I filed a bug/enhancement request in the Oracle bug database. As I was still working at Oracle back then and I had direct access to the bug database – primarily for the many beta testing activities I was involved in. The request was for an On Commit trigger, to fire immediately before the commit process would irrevokably be under way. The trigger should allow me to do some last minute business rule enforcements, to make sure the transaction was really, really okay.At the time I was working on Oracle’s CDM RuleFrame framework for the implementation of data oriented business rules. It was somewhat novel at that time, as it focused on transaction level business rule enforcement. Our philosophy was: as long as you do not commit, it does not matter one bit if the data is not in line with all data integrity rules. Who cares? At time you commit – that is when it should be okay. The kind of same idea behind the deferred declarative database integrity constraints. The problem we were facing is that we had no way to invoke this transaction level validation of rules when the application commits. The database will take care of ’simple’  declarative constraints, and for multi-table constraints or complex single table constraints that had to be programmed in PL/SQL, we could do nothing but enforce applications to perform this validation by deliberately invalidating a deferred check constraint from before statement level triggers for each DML type on all tables protected by our special business rules. Only if our CDM RuleFrame validation code had been called would we rectify that constraint violation and allow the transaction to continue. What I wanted to have was a way for the database to automatically invoke the CDM RuleFrame validation logic when the transaction is committed.The enhancement request never saw any action. So today, seven years onwards, we still do not have an On Commit trigger in the database. There some ways to emulate one, but none of them is ideal. In this article a brief report on what hooks we have for our make-believe On Commit trigger. The proceedings in the database surrounding the commit of a transaction include the following steps:validate deferred constraintsdelete all data from Temporary Tables with ON COMMIT DELETE ROWSsynchronize Materialized View with ON COMMIT REFRESHoperations on Materialized View Logs? Operations on Changed Data Capture Window?And once the transaction is really found to be okay:synchronize redo log buffers to filerelease locks ’submit’ pending jobssend alertsI have looked into these events to see which one could be used as a trigger for my own additional PL/SQL code.Deferred Constraints I had some hopes for Validation of Deferred Constraints. Well, that takes place but does not fire any triggers. I once thought that a Deferred Foreign Key Constraint with ON DELETE CASCADE in combination with a delete trigger on the referring (Child) table would work: the parent is deleted somewhere during the transaction, the ON DELETE CASCADE is part  of a deferred constraint and therefore takes place after the application has sent the commit command. Well, no: it turns out that even for deferred constraints, the cascade deletion of children is Immediate.Then I hoped that perhaps in 10gR2 the DML Error Log feature – which allows us to complete statements that violate declarative integrity constraints (see for example: http://orafaq.com/node/76) while writing the offending records to a logging table – would help out: an insert trigger on this logging table in combination with a deferred constraint that always gets violated during my transactions surely would do the trick? No, it does not, as DML Error Log does not work for deferred constraints…Note that when validation of deferred constraints runs into a violation, the entire transaction gets rolled back. There is an absolute minimum in terms of information on what violation for which record caused this rollback.Temporary TablesOne other event at commit time is the emptying of temporary tables with ON COMMIT DELETE ROWS set. That means that when the transaction commits, all records are flushed out of the TEMPORARY table. Well, I understood it would be naive to simply create a DELETE trigger on the temporary table (which I can do) and expect it to be fired if the records are removed at commit time (which it does not). The flushing of records from the temporary table is a low level, no log action – much like TRUNCATE. I still have to check whether perhaps a TRUNCATE system even level trigger would do the trick (though I doubt it).I though a smart approach would be: foreign key to temporary table with ON DELETE CASCADE and a delete trigger on the child table. However, it turned out that Referential Constraints to temporary tables are not allowed in the database. Another idea bites the dust. Refreshing Materialized ViewsWell, that means I am back to a solution I had found quite some time ago: the Materialized View. See for example the article Introducing Materialized Views as mechanism for Business Rule implementation (complex declarative constraints) for some background.  And more….?I still want to investigate if the changed data capture somehow gives me any hooks for event triggers. I should also investigate the TRUNCATE trigger on tempoary tables – though I doubt this approach: I am not sure if this event may be only after the commit itself, too late for my purpose. I am also not exactly clear on what Materialized View Logs exactly go through.If you would have any suggestions for mechanisms in the database that may help me get as close to a commit trigger as I can get, I would be thankful. […]

]]>
By: AMIS Technology blog » Blog Archive » Oracle 7Up to 10g - How time flies… Training for seasoned Oracle SQL and PL/SQL Developers https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1947 Fri, 14 Apr 2006 07:55:02 +0000 /?p=475#comment-1947 […] With the advent of the Oracle 7 RDBMS – 1993 – a new era dawned. Many of the key database functions and facilities, such as (Enforced) Integrity Constraints, Stored PL/SQL, Triggers, Snapshots, supplied packages and key SQL features were introduced in the Oracle 7 releases. Many Oracle developers also seem to be anchored in terms of knowledge and experience in the Oracle 7 database and the features and functions that release 7 had to offer. Even though we have moved on. And Oracle has moved on! It turns out that many very experienced developers are still developing in their Oracle 9i and 10g environments as though they live in the world of Oracle7. Every now and again they may accidentally make use of a more recent feature and of course they have read and heard about this and that new option, but Oracle7 is what’s at their fingertips.By the, not too long ago, I found myself in that exact same position. Programming SQL and PL/SQL in a very backwards compatible way! Realizing that predicament, I embarked on a program to study and take in the Oracle 8i, 9i and more recently 10g stuff that somehow had escaped my notice. This then turned into a workshop for my colleagues at AMIS, initially three days, now four very full days. This Oracle 7Up Workshop allows experienced SQL and PL/SQL Developers to get acquainted with (most of) everything that was added in Oracle 8i, 9i and 10g that is of practical use. It is not a marketing-style new features overview, it is an serious discussion of important, useful, timesaving, performance enhancing functionality. We quickly realized how this training would not only benefit the staff at AMIS: we also offered it to our business partners. By now we have trained a substantial number of software developers, including Oracle DBAs, Oracle Application Developers and Java developers who wanted to better grasp the full potential of SQL in an Oracle database. Most of if not all attendant were able to apply knowledge and code samples from the training in their own jobs starting the day after the training. Next week – on Thursday 20th april and Friday 21st April addressing SQL- and the week after – Thursday 28th and Friday 29th of April addressing PL/SQL- we have another edition of the Oracle 7Up Workshop (at our office in Nieuwegein, The Netherlands). And there are a few seats available for external participants. You can participate in one of the two blocks or attend both. Details on registration can be found here: AMIS Trainingen. Topics in the 7Up Training Now what are all these grand new features that we should know about, I hear you ask. Well, some of the major ones in the SQL session are: Analytical Functions, Multi-Table Insert and Merge, In Line Views, FlashBack Queries, Materialized Views, Oracle Text, Advanced Constraints implementation, Database Design (Index Organized Tables, Temporary Tables, Function Based Indexes, Instead Of Triggers), Join Syntax, Hierarchical Queries.The PL/SQL session discusses topics like: Collections and Object Types, bulk operations, dynamic SQL, fine grained access control and auditing, autonomous transactions, system event triggers, new PL/SQL functions and syntax. For more details and examples of the slides and the labs used in this workshop, see Oracle 7Up Training – Sample of Materials.Discussion of "7Up Topics" on the weblogMany of topics that we discuss in the 7Up training have been subject of one or more articles on our weblog. Recent examples of such articles include:Default Column Value – the syntax for defining default values in your table definitions is richer than you probably realize, including the option to dynamically set default values, depending on the context How using ROWNUM in In-Line Views can speed up your query – ROWNUM seems like a pretty innocent, not very exciting ‘feature’  of SQL Queries. However, including ROWNUM in a query can have pretty drastic effects – and sometimes positve ones – on your query performanceFunctions NVL2 and NULLIF – some new functions that can make constraint definitions and where clauses more compactFinding overlap between time-periods using LAG and LEAD – Analytical functions allow you to look forward or backward in a set of rows, reducing the need for many complex joins!Materialized view as mechanism for the declarative (PL/SQL free) implementation of complex business rules – Declarative implementation of business rules like Master must have a restricted number of details (no fewer than X and not more than Y), The Sum of a certain Detail Column is restricted within a master and The start-date and end-date of detail-records must be between the Master’s Startdate and Enddate: it can be done!How we can edit files through simple insert and update operations – Peter Kok’s article on Updateable External TablesRapidly building a WebSite search engine using Oracle Text – Investigating Bill of Materials – Juggling with tree-structures and hierarchical queries using Oracle 9i and 10g Connnect By enhancementsAspect Oriented Programming in PL/SQL – Making surprising use of the After Create (on compile) TriggerMerge! – a real li(f/v)e saver and many more…. […]

]]>
By: Lucas https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1946 Sun, 16 Oct 2005 06:35:03 +0000 /?p=475#comment-1946 For more explanation around this use of Materialized Views, seehttp://www.dbazine.com/oracle/or-articles/tropashko8 for an interesting discussion about the same topic. It discusses rules such as Unique Key Constraint Spanning Multiple Tables and Cardinality Constraints.

]]>
By: Alex Nuijten https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1945 Tue, 28 Jun 2005 06:17:56 +0000 /?p=475#comment-1945 Hi Shafi,
Do you have SUM and COUNT columns in your TEMPTRAN table? Or is this a typo and did you want something like:

CREATE MATERIALIZED VIEW MV_TEMPTRAN
REFRESH FAST ON COMMIT
AS
SELECT LEDGERCD, SUBLEDCD, BRCD, VALUE_DATE, VCHCODE,
SUM (BALANCE), COUNT (CNT), --< ------ Note the parenthesis
COUNT (CNTAMT) --<------- Note the parenthesis
FROM TEMPTRAN
WHERE authorised = ‘Y’
GROUP BY LEDGERCD, SUBLEDCD, BRCD, VALUE_DATE, VCHCODE;

(I assume that BALANCE, CNT, CNTAMT are columns in your TEMPTRAN table…)

]]>
By: Shafi https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1944 Mon, 27 Jun 2005 10:16:39 +0000 /?p=475#comment-1944 While creating materialized view on single table with the following conditions?
I am getting error, any solutions?

SQL>CREATE MATERIALIZED VIEW LOG ON TEMPTRAN WITH
ROWID, (LEDGERCD, SUBLEDCD, BRCD, VALUE_dATE, VCHCODE)
INCLUDING NEW VALUES;

sql>CREATE MATERIALIZED VIEW MV_TEMPTRAN
REFRESH FAST ON COMMIT
AS
SELECT LEDGERCD, SUBLEDCD, BRCD, VALUE_DATE, VCHCODE,
SUM(VCH_AMT_MV) BALANCE, COUNT(*) CNT,
COUNT(VCH_AMT_MV) CNTAMT
FROM TEMPTRAN
WHERE authorised = ‘Y’
GROUP BY LEDGERCD, SUBLEDCD, BRCD, VALUE_DATE, VCHCODE;

ORA-12054: cannot set the ON COMMIT refresh attribute for the materialized view

]]>
By: Roger Wienen https://technology.amis.nl/2005/03/24/introducing-materialized-views-as-mechanism-for-business-rule-implementation-complex-declarative-constraints/#comment-1943 Mon, 28 Mar 2005 20:49:33 +0000 /?p=475#comment-1943 Ik onderscheid twee soorten beperkingsregels, die ieder op hun beurt weer verder onderverdeeld kunnen worden. Allereerst de regels die een niet-toegestane toestand (NTT) beschrijven en als tweede de regels die een niet-toegestane overgang (NTO) beschrijven.

En niet-toegestane overgang is de combinatie van één toegestane begintoestand en één toegestane eindtoestand. Een toestand is een stelsel van feiten die op één moment in de tijd waar zijn.

Een voorbeeld van een NTT:

1. Persoon met sofi-nummer 1560.91.642 is geboren op 07-07-1970
2. Persoon met sofi-nummer 1560.91.642 is geboren op 31-07-1968

Deze twee zinnen kunnen niet tegelijkertijd waar zijn. Een persoon kan niet op twee verschillende tijdstippen geboren zijn. Daaruit kan men afleiden dat het sofi-nummer een unieke identificatie is voor een persoon. Dat resulteert in een primaire of alternatieve sleutel.

Een voorbeeld van een NTO is :

Begintoestand (TT1)
1. Persoon met sofinr 1560.91.642 ontvangt een AOW uitkering

Eindtoestand (TT2)
2. Persoon met sofinr 1560.91.642 ontvangt een WW uitkering

Een persoon mag geen WW uitkering ontvangen nadat hij/zij een AOW heeft ontvangen.

De NTT kunnen in een relationele database worden gecontroleerd met SQL. In bovengenoemd geval betekent dat de vraag “Welke personen zijn geboren op meer dan één tijdstip� niet beantwoord mag worden, c.q. een lege verzameling moet zijn. Indien dat wel het geval is, krijgt de desbetreffende gebruiker in het gunstigste geval de melding welke (combinatie van) zinnen welke beperkingsregel overtreden.

De meeste databases hebben de mogelijkheid bepaalde type beperkingsregels declaratief vast te leggen, dus zonder gebruik te maken van SQL.

De NTO kunnen alleen met SQL worden gecontroleerd indien er historische informatie opgeslagen is, zodat men de feiten van de begintoestand kan reproduceren. In bovengenoemd geval betekent dat de feiten als volgt beschreven moeten worden:
1. Persoon met sofinr 1560.91.642 ontvangt als eerste een AOW uitkering
2. Persoon met sofinr 1560.91.642 ontvangt als tweede een WW uitkering

en men dus een NTT beschrijft.

Indien dit niet gewenst is, kan men dit met behulp van database triggers controleren.

De diverse feiten worden achtereenvolgens bekend gemaakt aan de database. In het geval dat de zinnen in diverse administratiestructuren (tabellen) vastgelegd worden, kan de controle alleen achteraf, dus nadat alle relevante feiten bekend zijn gemaakt. Dat betekent dat er meerdere statements nodig zijn. Ik miste altijd de mogelijkheid zaken te controleren op transactie-niveau, zeg maar een PRE-COMMIT trigger. Voor de afleidingsregels (of rekenregels) mis ik nog een POST-COMMIT trigger.

Door op deze creatieve wijze gebruik te maken van de mogelijkheden van de Oracle database, betekent dit dat elke beperkingsregel op deze wijze gecontroleerd kan worden. En zodoende de (niet officiële) PRE-COMMIT trigger is geboren.

Ik mis inderdaad nog de mogelijkheid om de gebruiker mede te delen welke regels overtreden zijn en welke combinaties van zinnen hieraan ten grondslag liggen.

De mooiste user interfaces sluiten overtredingen uit. De op één na mooiste user interfaces geven de gebruikers de mogelijkheid de algemene regel te kennen en correctief op te treden, dus de zinnen aan te passen die de regel overtreden.

]]>