10 tips for ADF Application (Module Pool) tuning

Performance tuning is one of the things that is often left behind during development. Project leaders and business owners seem to think it happens on its own or just push it to the background when the deadline comes close. Another thing that doesn’t help is that performance tuning for ADF applications is a more advanced topic. During a regular ADF training there’s so much to talk about there simply isn’t enough time to cover it, so people are left on their own to try and figure out how it works. There are also a lot of things that have an influence on performance like business demands and application user experience which may not be directly obvious. In this post I will give you 10 great tips on tuning to give you a starting point.

1. AM Pool tuning != Performance test one month before launch.
   Often a performance or load test is scheduled just before launch, even after testing is complete. I see this a lot. You don’t want to do this. If you do find issues with performance it may well be that there are some fundamental things wrong with your application that you cannot fix and retest within the given timeframe. Your fix may even break performance or functionality in other places. In 90% of the cases where it is done this way you end up with your application in production anyway because of the deadlines causing a lot of users to be unhappy. Which does equal unhappy management.
2. Start early in the development phase
   pretty much follows out of bullet number 1. Start early so when you do have wrong settings on VO’s or on the AM you don’t have to refactor the whole application and regular testing will tackle any issues you may introduce. Changing how your application is build is also a lot easier if it’s still small.
3. Get !someone! to actually tell you what you are tuning for
   This is actually a lot harder than it sounds. I still see a lot of applications where the non-functional requirements, if there are any, hardly ever include anything on security or performance and expected user statistics. This information coupled with the expected amount of data that changes and gets stored is essential to how you tune your application. The numbers on data and it’s usage tell you something on how to organize your application and the numbers on user tells you how to set the numbers in your AM configuration. Try and get numbers on max logged in users, numbers of concurrent users, session times, how usage and load is spread across your application.
4. Test with pooling off as well as with everything on 1
   Testing your application with AM pooling turned of is reasonably well known although it will get skipped due to time constraints and negligence if you’re not careful. One way around this is to use the option to use JVM parameters on a server to turn it on and off. Another less well known thing is that you want to test with everything set to 1 as well. This to emulate the fact that an AM is going to be reused by another user. This way you can check to see that there isn’t any state being saved that is getting used by another person’s session. 
5. Check for large passivations
   Normally the database is used as the passivation store for an application. ADF is saving the state in a table called PS_TXN. It doesn’t hurt to have a looksee in this table with a query that gives you the size of the lob inside. You want the numbers you see in this query to be as low as possible because all of that data needs to go over the line on an activate/passivate. When you find a record that is extraordinarily large you can have a look at that record and the blob to see what is causing it. Generally a VO with a big resultset and an attribute that has the passivate checkbox on. What also helps is to add the following class to your ADF logger on finest: oracle.jbo.server.Serializer. This is the class responsible for serializing the AM and on finest it will print the blob that will go to the database to the log and thus give you direct insight in what is happening during development.
6. Generally speaking… Less state is better
   This one combines nicely with the previous point. State pretty much complicates anything. AM state will slow down your activations and passivations as well as consume server memory. State in beans will add to the memory load on the server. More memory on the server will cause more garbage collects slowing your application and frustrating your users. The manual says to use the narrowest scope possible, it is right.
7. Use release states on your Application Module
   This one is overlooked a lot. Let me elaborate a little bit. An Application Module can have several states when it is in the pool: available with reference, available with no reference, and in use. When an AM gets assigned to a user it holds a reference to that user and it’s state. When that AM gets assigned to another user the AM will passivate the state to another user and activate any state that user may have. An AM with no reference has no state to safe and will therefore be faster. The release state of an AM determines if it gets put back in the the pool with a reference. We’ve already seen that big activations/passivations are bad for business. Keeping application state can be really handy, but if you don’t need it anymore get rid of it. Better yet if you don’t need state don’t create it. Default your Application Module is put back in the pool with managed level. Making sure you release your Application Module with the unmanaged level makes sure the framework understands it doesn’t need to save any state for later use. Good for a lot of other points above, so use it whenever there’s no functional need to save further state. The easiest way to do so is to call the resetState method on your datacontrol.
8. Do the occasional memory dump so you know where the memory is going
   It doesn’t hurt to do the occasional memory dump of the JVM every now and then. With a good tool like MAT (Eclipse Memory Manager) you can see very detailed what your memory is being used for. This will give valuable insight in possible tuning options and will give you an indication on the amount of memory consumption when you’re application goes to production. Also don’t forget things like JRockit flight recordings to find out what your JVM is up to.
9. Don’t forget your application is only a small part of a chain
   You can tune your application all you want, but you need to keep in mind that pretty much every setting you tweak will have effect on your Application Server, Database and the OS everything runs on. The settings that govern the amount of AM’s in your pool also affect the number of connections that need to be available in your AS connection pool as well as database sessions and for Linux the number of files that are allowed open. The way your queries are build, the amount of records in search results, datamodel, etc., will effect memory on the AS, DB and therefore OS as well. The main point here is don’t solely focus on your application but keep in mind the effects on other components of the infrastructure as well.
10. Tuning isn’t finished after Go Live is successful
    Off course tuning isn’t finished after your application. Chances are high that quite a few settings where made on assumptions on amounts of users and how they use your application. If you’re lucky you had a load test but monitoring after launch is always good idea. Your assumptions may have been wrong, users may have different or changed behavior, amounts of user may fluctuate, data within the database may change causing queries to become inefficient, etc. Become friends with the resident DBA (always a good idea, they know stuff) and check the AWR reports. Keep making the occasional Flight recording and memory dump to ensure all is well. Last but not least make sure some form of automated monitoring is in place and alerts are send at a point where you can recover without total system shutdown.