Yesterday I had the opportunity to present my developed performance profiling approach called â€œGAPPâ€ at the HOTSOS Symposium 2008 here in Dallas. â€œGAPPâ€ is an abbreviation of â€œGeneral Approach Performance Profiling and is based on data mining of all kind of gathered system statistics (but also other data is possible).
The presentation went well and a lot of people attended my presentation. I got a lot of nice criticism and was questioned a lot by Dr. Neil Gunther. Before and after the presentation I had a lot of nice discussion with him and he even offered to help me make this approach even bigger. For me this was a real honor and I really was very happy with his input.
Also a lot of other â€œimportantâ€ people attended the presentation, like Anjo Kolk, James Morle, Cary Millsap, Jeroen Evers, Toon Koppelaars and many others. I personally was very happy to have the chance to present for such an audience, and was very happy with their reactions and criticism. I personal think that the HOTSOS Symposium is â€œThe Placeâ€ for performance in the world.
Dr. Neil Gunther sitting with me:
To develop â€œGAPPâ€, it took a long time to come up with a way to be able to have all kind of metrics (CPU, I/O, etc.) able to say something about a business process response time. I already started to think about this when I was still working for Oracle, and later on when I was working for IBM. After a lot of hours, from which my wife said: â€œWhat you are doing again behind your laptop?â€ I have now the feeling that the approach is worth having a name.
To give an impression of where â€œGAPPâ€ is all about, I will give a small introduction to the approach I created. First of all the approach can be used to performance profile business processes on high level, but depending on the metrics you put in, even on detailed level. If you look at the below slide from the presentation it will become clearer:
In this slide it becomes obvious that â€œGAPPâ€ for now (maybe in future I can go further), can say something about the fluctuation of â€œWait Timeâ€ for each resource in a chain for a business process. The â€œRâ€ and the â€œRslowâ€ in the slide are only different due to the â€œWait Timeâ€ for the different resources (CPU, I/O, Memory, etc.). Further I use the fact that utilization of a resource is related to â€œWait Timeâ€.
By having system metrics from all machines in a chain of an architecture, and having the response time of the business process on the same timestamps as the system metrics, the approach makes it possible to clarify which system metric is influencing the business process response time the most (Factorial Analyses). To accomplish this job on the moment I used a package called â€œDBMS_PREDICTIVE_ANALYTICSâ€, which is part of Oracle Data Mining (ODM), but also another data mining tool could be used (open source maybe?).
After knowing which system metric is influencing the business process the most, you could start to focus on that part of the architecture and do your optimization on the best spot to start with. To give an impression the following graph gives an impression of the â€œFactorial Analysesâ€. In the presentation I used an architecture with two machines (â€œEâ€ and â€œQâ€):
In this graph it becomes clear that â€œWait%â€ and â€œDisk Busy%â€ of logical hard disk 2 and 3 from machine â€œQâ€ explains the most response time fluctuation of the business process in the months mentioned in the slide. Further investigation should be done on machine â€œQâ€. Later in the presentation I have showed that in other periods the â€œEâ€ machine should be investigated.
An important remark is that the approach â€œGAPPâ€ can work completely independent from Oracle, it can also use completely different metrics like â€œSQL statementâ€ elapsed times. This makes it even possible to find out if a certain SQL statement is highly involved in the response time of a business process.
Also future response time prediction can be made by â€œGAPPâ€. The effect can be predicted, how much changes in certain metrics will impact the response time of the business process.
I would love to clarify more, but I think I will need to create an article to show how powerful this approach â€œGAPPâ€ actually is. In the end I want to say that for me I really â€œJumped over the GAPPâ€ I had in my data to analyze the performance problem I was facing at the customer. The longer I look at the approach, the more I find out that this approach is really very powerful, although still a lot of research should be done.
I thank everybody who attended my presentation at the HOTSOS Symposium 2008 and of course at the AMIS Query in the Netherlands. If you have any questions about the approach I would be glad to answer any question about it (firstname.lastname@example.org). Further I would like to thank Cary Millsap for giving me the chance to present at the HOTSOS Symposium 2008.
Of course I will keep you posted about the progress of the approachâ€¦ â€œGAPPâ€
With kind regards, Gerwin