Sharing only the APPL_TOP in 11.5.10

In the past days I have implemented a well known architecture to many of my fellow Oracle E-Business Suite DBAs:
An architecture that consists of multiple Application Servers.

Until version 11.5.10, the common practice was to share the APPL_TOP between multiple Application Servers, and from 11.5.10 onwards Oracle started supporting the sharing of the ORA_TOPs as well. This solution is called the Shared Application Tier File System. You can read a lot about this new feature of 11.5.10 on Oracle Metalink.

However, with the introduction of the support for the shared Technology Stack (8.0.6 and iAS ORACLE_HOME), Oracle decided to push this to the market in such a way that customers who don’t want to share the Technology Stack are left without directions on how to get there.
Let there be no misunderstanding: sharing only the APPL_TOP and not the Technology Stack is still supported, even in 11.5.10 and higher releases. The problem is that there is no documentation about such an architecture.

My colleague Bas Klaassen told me that he had tried such an architecture using rapidwiz, and didn’t succeed finishing the rapid install, because it wouldn’t accept sharing the APPL_TOP, and putting the Technology Stack onto a private file system.
When I started this migration, I had a single instance, multi-tier (one database and one application server) EBS environment to clone from, so this particular issue wouldn’t hit me anyway.

Here is how I did the migration from single instance, multi-tier to a 2-node RAC with 2 Application Servers, sharing only their APPL_TOP:

• The first thing to do was to run adpreclone on the source environment, both on the database and on the application server
• Next, I copied the database to my target system, which was already utilized with 10gR2 RAC.
• Then I copied the APPL_TOP and COMMON_TOP to the target application server(s) onto the shared file system.
• The next step was to copy the Technology Stack to the target application servers onto private file systems (on each node)
• Now, having everything in place I took a wrong decision…
  I decided to first clone the database server, leaving it to a non-RAC database for the moment. Huh? wrong decision? Yes, wrong decision. I will get back to that later. The result was a newly configured EBS Database Server, fully functional with autoconfig and all. No problem, for now anyway.

The next thing I did was to run adcfgclone on the application tier. Since I was planning to configure multiple application server nodes, I replied appropriate to the question whether this environment consists of multiple application server nodes (Y, duh!).
Anyway, this all went fine. The application server processes were started, and EBS was up and running, however on one application node still.
I ran adcfgclone on the second application server; same story, no problems encountered, everything was up and running with two application servers.

Following the whitepaper on metalink about Configuring EBS with 10gR2 RAC and ASM, I now decided to convert my database to RAC using rconfig. After editing the convertToRAC.xml (in ORACLE_HOME/assistants/rconfig/sampleXML), I ran rconfig. Can you imagine my surprise when at the end of rconfig the thing crashed with an error: ORA-00439: feature not enabled: Real Application Clusters.

All of a sudden, I couldn’t start my database in clustermode. What happened? The adcfgclone for the database tier was run against a non-RAC database; in fact, I decided (the wrong decision I was talking about) to leave the database in single-instance mode for the moment, so I ran adcfgclone and answered no to the question whether this was a RAC database. adcfgclone in its turn, relinks the executables in the ORACLE_HOME and turns off the Real Application Clusters feature.
It wasn’t a big issue, luckily. I just copied the ORACLE_HOME from the other RAC node back to the first, recovered the appsutil and TNS directories from the damaged ORACLE_HOME, and tadaa, the Real Application Clusters feature was back in business. Another run of rconfig (after restoring the original database) and everything was ok.

Enabling autoconfig for the database (which was now in RAC) wasn’t too hard.

• First create a new xml-file (adbldxml),
• then run adconfig.pl from the database tier, which will fail (definetly!),
• stop the database,
• stop the listener,
• source the newly created environment file,
• start the listener (from $TNS_ADMIN! the location will be changed after running adconfig!),
• start the database,
• run adconfig again.

Now it was time to configure my application tier for Real Application Clusters. No problem, just run “adautocfg” from the $COMMON_TOP/admin/scripts/<context_name> directory and voila, the first Application Server Node was a fact. Run the same routine on the second node and we are running with two Application Server Nodes.

That was easy! The problem only is that with 11.5.10 there is no documentation from Oracle that provides these steps. In fact, this method I used, is exactly the same as I would (and should) have done with 11.5.9. There really is no difference.