SOA Suite can sometimes be a bit slow to start. This is especially the case when there are a lot of composites to load. Customers using different versions of composites can benefit from undeploying non-default revisions of processes which do not have any running instances (see for example here). Undeployment in most cases is an asynchronous process which does not give feedback. It can partially fail without you noticing (apparently not an atomic transaction). This sometimes leaves composite remains; parts of the composite which are still loaded at startup but are not visible from the Enterprise Manager. Removing these can dramatically reduce server startup time. Especially in an environment which has been used for some time and environments with many versions of composites. Reducing the time required to get the soa-infra application fully up and running is of course mostly relevant for 11g SOA installations and less for 12.1.3 (which does some lazy loading) and 12.2.1 (which supports parallel deployments, also during server start-up).

In this article I’ll demonstrate how these left-over composite parts can be identified and removed on an 11.1.1.7 SOA environment. First try this procedure on a development or test environment before executing it in production! Using this method is entirely at your own risk. Only try it if you understand what you are doing. If something breaks, tell me so I can update this article. Thanks!

Please mind that these actions, although they help with the start time and memory usage of your SOA environment, have less impact on run-time performance than for example purging of instances and reducing the amount of deployed composites (or tweaking datasources, soa-infra database, JVM, etc).

Cleaning

What is a composite in the MDS?

WebLogic Server first brings up its Managed SOA server. After this managed server is up and RUNNING, the soa-infra application is started. After the soa-infra application is started, the composites are loaded from the MDS and put in memory.

There is a folder called ‘deployed-composites’ in the root of the soa-infra MDS. If you are an experienced Oracle SOA developer or administrator, you have probably encountered this folder before. For example when the soa-infra application does not want to start, you can remove an entry in deployed-composites.xml which is in the deployed-composites folder (see here. you can even do this offline (here)). A composite however is not only an entry in the deployed-composites.xml file. There is also a separate folder which contains the files of which the composite consists. These are mostly XML and JAR files.

deployed-composites

Entries in the deployed-composites.xml file are visible in the Enterprise Manager. Entries which are not in the deployed-composites.xml file but do have a folder in the MDS, are not visible, but are loaded at server start. This increases startup time and memory usage.

The below procedure can be performed on a running SOA server. Do mind though that the effects will only be visible after a server restart. Please mind to clean the WebLogic cache folder after a server shutdown and before a server start (user_projects/domains/yourdomain/servers/yourserver/tmp) to make sure you start fresh. The first time after cleaning the cache folder, the server start might take a while because it needs to fetch everything again from the MDS. The second time it will be faster.

Cleaning procedure

First create an MDS export. This is described here. When you have this export, you can extract it and execute the following Python 2.7 script (don’t forget to update the paths for your environment and the SOA server name). What the script does is look at the deployed-composites.xml file and identify all entries in the default partition of the deployed-composites folder which do not have an entry in deployed-composites.xml. Next it creates a WLST script to remove these entries. The WLST script which is created, should have a ‘connect’ statement at the top and should be executed against your SOA (managed) server. You can also execute it remotely. Make sure you pick the correct wlst.sh or wlst.cmd to start a WLST interpreter which has deleteMetadata available. deleteMetadata was the only WLST command I found which allows deleting documents outside the ‘apps’ MDS folder.

 import os   
 import xml.etree.ElementTree as ET   
 import xml.dom.minidom as minidom   
 import sys,re   
 import argparse   
     
 deployed_composites="D:\\tmp\\soa-infra_metadata\\deployed-composites\\deployed-composites.xml"  
   
 #read a file and return a ElementTree   
 def get_tree_from_xmlfile(filename):   
   if os.path.isfile(filename):   
    tree = ET.parse(filename)   
    return tree   
   else:   
    raise Exception('Error opening '+filename)   
   
 def get_deployedcomposites(deployed_composites):  
       deployed_composites_root = deployed_composites.getroot()  
       return deployed_composites_root  
         
 deployed_composites_el = get_tree_from_xmlfile(deployed_composites)  
 deployed_composites_root = deployed_composites_el.getroot()  
   
 list = []
 for rev in deployed_composites_root.findall('composite-series/composite-revision'):  
      list.append(rev.attrib.get('dn').replace('!','_rev'))  
 for dir in os.listdir('D:\\tmp\\soa-infra_metadata\\deployed-composites\\default'):  
   if 'default/'+dir not in list:  
           print "deleteMetadata(application='soa-infra',server='soa_server',docs='/deployed-composites/default/"+dir+"/**')"

Directories

I did not find a way to delete directories outside the apps folder by using WLST commands. Luckily the Oracle A-Team also realized this and wrote some custom Java code to address this issue. See here. The blog article offers a project which you can download which contains sample code on how to programatically do interesting things to the MDS. For this specific use-case I updated the code to look for directories which do not contain documents. These directories recursively removed (that’s why I have to catch a NullPointerException. the directory could already be gone). Of course first check if this code works for you before actually deleting directories. I only updated the work method in the sample code provided by Oracle which you can download here. You can download my altered method here

Be careful with this code if you use other things inside the MDS than documents and packages (folders) such as customizations.

The MDS schema

Everything which was present in the MDS once, but has been deleted, remains in the MDS database. You can confirm this by looking at the SOA_MDS schema and browse for items you thought had been deleted in the MDS_PATHS table. Deleted items can be identified by checking MDS_PATHS for entries which do not have a PATH_HIGH_CN which is null. See here. Deleted documents however are not loaded by the server during startup, but if you once had a lot of stuff in the MDS, the below procedure might be worthwhile since MDS queries will perform faster with less data.

Manually ‘fiddling’ with the data in the SOA_MDS tables is not recommended and dangerous since there are no foreign key constraints but there are references from other tables! (with column names which do not easily allow you to identify the links between data in the different tables). As a workaround, you can create an MDS export (this only takes the ‘not-deleted’ documents and folders), backup the SOA_MDS schema, truncate tables containing documents/folders (not the partitions and similarly important stuff!) and re-import the MDS export. You can also recreate the SOA_MDS schema completely with the RCU (Repository Creation Utility) before the import. This should provide you with a clean MDS. As indicated before, do test this procedure to make sure you have identified the correct SOA_MDS tables! There are also several other ways to tweak MDS performance. See here.