The Oracle SOA Suite infrastructure, especially composites, use the database intensively. Not only are the process definitions stored in the database, also a lot of audit information gets written there. The SOA infrastructure database, if not well managed, will grow and will eventually have detrimental effects on performance. In this blog post I will give some quick suggestions that will help you increase performance of your SOA Suite infrastructure on the database side by executing some simple scripts. These are some suggestions I have seen work at different customers. Not only do they help managing the SOA Suite data in the database, they will also lead to better SOA Suite performance.
Do not log too much!
Less data is faster. If you can limit database growth, management becomes easier.
- Make sure the auditlevel of your processes is set to production level in production environments.
- Think about the BPEL setting inMemoryOptimization. This can only be set for processes that do not contain any dehydration points such as receive, wait, onMessage and onAlarm activities. If set to true, the completionpersistpolicy can be used to tweak what to do after completion of the process. For example only save information about faulted instances in the dehydration store. In 12c this setting is part of the ‘Oracle Integration Continuous Availability’ feature and uses Coherence.
Start with a clean slate regularly
Especially for development environments it is healthy to regularly truncate all the major SOAINFRA tables. The script to do this is supplied by Oracle: MW_HOME/SOA_ORACLE_HOME/rcu/integration/soainfra/sql/truncate/truncate_soa_oracle.sql
The effect of executing this script is that all instance data is gone. This includes all tasks, long running BPM processes, long running BPEL processes, recoverable errors. For short everything except the definitions. The performance gain from executing the script can be significant. You should consider for example to run the script at the end of every sprint to start with a clean slate.
Delete instances
Oracle has provided scripts to remove old instances. These are scheduled by default in a clean installation of 12c. If you upgrades from 11g to 12c, this scheduling is not enabled by default. The auto-purge feature of 12c is described here.
What this feature does is execute the standard supplied purge scripts: MW_HOME/SOA_ORACLE_HOME/rcu/integration/soainfra/sql/soa_purge/soa_purge_scripts.sql
In a normal SOA Suite 12c installation you can also find the scripts in MW_HOME/SOA_ORACLE_HOME/common/sql/soainfra/sql/oracle
In 12c installations, the patched purge scripts for older versions are also supplied. I would use the newest version of the scripts since the patches sometimes fix logic which can cause data inconsistencies which can have consequences later, for example during migrations.
What the scripts do is nicely described here. These scripts only remove instances you should not miss. Running instances and instances which can be recovered, are not deleted. In the script you can specify for how long data should be retained.
You should schedule this and run it daily. The shorter the period you keep information, the more you can reduce your SOAINFRA space usage and the better the performance of the database will be.
An example of how to execute the script:
DECLARE
MAX_CREATION_DATE TIMESTAMP;
MIN_CREATION_DATE TIMESTAMP;
BATCH_SIZE INTEGER;
MAX_RUNTIME INTEGER;
RETENTION_PERIOD TIMESTAMP;
BEGIN
MIN_CREATION_DATE := TO_TIMESTAMP(TO_CHAR(sysdate-2000, ‘YYYY-MM-DD’),’YYYY-MM-DD’);
MAX_CREATION_DATE := TO_TIMESTAMP(TO_CHAR(sysdate-30, ‘YYYY-MM-DD’),’YYYY-MM-DD’);
RETENTION_PERIOD := TO_TIMESTAMP(TO_CHAR(sysdate-29, ‘YYYY-MM-DD’),’YYYY-MM-DD’);
MAX_RUNTIME := 180;
BATCH_SIZE := 250000;
SOA.DELETE_INSTANCES(
MIN_CREATION_DATE => MIN_CREATION_DATE,
MAX_CREATION_DATE => MAX_CREATION_DATE,
BATCH_SIZE => BATCH_SIZE,
MAX_RUNTIME => MAX_RUNTIME,
RETENTION_PERIOD => RETENTION_PERIOD,
PURGE_PARTITIONED_COMPONENT => FALSE);
);
END;
/
The script also has a variant which can be executed in parallel (which is faster) but that requires extra grants for the SOAINFRA database user.
Shrink space
Tables
Deleting instances will not free up space on the filesystem of the server. Nor does it make sure that the data is not fragmented over many tablespace segments. Oracle does not provide standard scripts for this but does tell you this is a good idea and explains why here (9.5.2). In addition you can rebuild indexes. You should also of course run a daily gather statistics on the schema.
For 11g you can use this script to shrink space for tables and rebuild indexes. You should execute it under XX_SOAINFRA where XX if your schema prefix.
LOBs
LOB columns are saved outside of the tables and can be shrunk separately. In the below script you should replace XX_SOAINFRA with your SOAINFRA schema. The script explicitly drops BRDECISIONINSTANCE_INDX5 since the table can become quite large in development environments and you cannot shrink it with the index still on it. This script also might overlap with the script above for tables with LOB columns. It only shrinks for large tables where the LOB columns take more than 100Mb of space.
Other database suggestions
Redo log size
Not directly related to cleaning, but related to SOAINFRA space management. The Oracle database uses so-called redo-log files to store all changes to the database. In case of a database instance failure, the database can use these redo-log files to recover. Usually there are two or more redo-logfiles. These files are rotated: if one is full, it goes to the next. When the last one is filled, it goes back to the first one overriding old data. Read more about redo-logs here. Rotating a redo-log file takes some time. When the redo-log files are small, they are rotated a lot. The following provides some suggestions in analyzing if increasing the size will help you. I’ve seen default values of 3 redo-log files of 100Mb. Oracle recommends having 3 groups of 2Gb each here.
Clean up long running and faulted instances!
The regular cleaning scripts which you might run on production do not clean instances which have an ECID which is the same as an instance which cannot be cleaned because it is for example still running or recoverable. If you have many processes running, you might be able to win a lot by for example restarting the running processes with a new ECID. You do have to build that functionality for yourself though. Also you should think about keeping track of time for tasks. If a certain task is supposed to only be open for a month, let it expire after a month. If you do not check this, you might encounter large numbers of tasks which remain open. This mains the instance which has created the task will remain open. This means you cannot undeploy the version of the process which has this task running. Life-cycle management is a thing!
Finally
SOAINFRA is part of the infrastructure
Oracle SOA Suite logs a lot of audit information in the SOAINFRA database. You might be tempted to join that information to other business data directly on database level. This is not a smart thing to do.
If the information in the SOAINFRA database is used to for example query BPM processes or tasks, especially when this information is being joined over a database link to another database with additional business data, you have introduced a timebomb. The performance will be directly linked to the amount of data in the SOAINFRA database and especially with long running processes and tasks. You have now not only introduced a potential performance bottleneck for all your SOA composites but also for other parts of your application.
It is not a system of records
Secondly, the business might demand you keep the information for a certain period. Eventually they might even want to keep the data forever and use it for audits of historic records. This greatly interferes with purging strategies, which are required if you want to keep your environment to have good performance. If the business considers certain information important to keep, create a table and store the relevant information there.
Great article.