First step with Oracle Management Cloud – Application Performance Monitoring, mandatory for DevOps

A few days back, we at AMIS got our cloud trial for Oracle Management Cloud. I can now report from my first steps with Application Performance Monitoring, one of the key components of OMC. Application Performance Monitoring (APM) is clearly indispensable to any organization adopting a DevOps approach – and frankly required for any organization in general running applications to support business objectives. APM provides insight in the non-functional behavior of applications – or better yet: of the business functions provided by these applications. It alerts administrators to functions that have unacceptable response times or are at risk to display poor performance and it allows us to analyze these situations to figure out where in the application stack – front end, services, integration flows, database, etc. – and in which specific component the problems have arisen. After performing this type of root cause analysis, resolving the problem still needs to be done, but is kick started as early as possible and with as much analysis details as possible.

In this article, I will describe the very practical steps I took to go from having my trial provisioned to having my first application monitored in the dashboards of APM. At this point I will not yet have very compelling analyses to describe – but I do have a dashboard and my first alerts sent to me.

This picture visualizes my set up: a local Node.js environment on my laptop, running an application that responds a HTTP requests by serving up an Oracle JET application with all of its resources (static JS libraries, CSS files, images etc). The Node.js server is configured for oracle-apm as is this particular Node.js application. When the application is running, the APM agent is activated. The agent gathers details and metrics – and sends these details to the Oracle Management Cloud environment. Here, these details are collected, stored and processed. They can be visualized in a dashboard, used for analysis and for example lead to automated alerts when specified alert conditions are identified.

 

Let’s take a look at the steps I took to get my pre-existing laptop, Node.js environment and application hooked up with APM. Note: most steps come straight out of the documentation – except for two that I will mention explicitly.

Enter the Set Up Wizard for Application Performance Monitoring; note: this wizard appears when you access the APM console for the very first time. At a later time, you can open the download dialog from the Application Navigator, which can be accessed through the icon in the top bar, like this:

Here is the initial dialog:

:

On page two, you can download the Master Installer. Also take a note of the registration key (you can find the key later on as well)

Downloading the installer:

And extracting its content to a staging directory – d:\omc in this case:

Configure Node.js for APM

Set the STAGE_DIR and NODE_HOME environment variables, to the extraction directory (d:\omc) and the installation directory of the local Node.js instance (“D:\Program Files\nodejs”).

Install the agent from the staging directory where it was extracted to:

d:\omc\AgentInstall.bat AGENT_TYPE=apm_nodejs_agent STAGE_LOCATION=d:\omc AGENT_REGISTRATION_KEY=RMxMm7chywi-J-VZ7_

The staging directory should contain some additional files along with the other APM Node.js Agent installation content:

From the staging directory, execute the ProvisionApmNodeAgent.cmd file:

This  creates oracle-apm as a global node module under the the node_modules directory of the Node installation on Windows: <USER_HOME>\AppData\Roaming\npm\node_modules as shown below.

The oracle-apm, directory contains  a folder called data with two files:

 

Annotating a Node.js application to monitor

Before a Node.js application will start leveraging the APM agent to generate metrics and send these to the OMC cloud environment, the application has to be prepared – in a very minimal way. The main entry point to the application – the file that is executed when the application is started – has to require the oracle-apm module. Here I have a Node.js application that starts by running the bin/www file. This file is extended with one line:

require('oracle-apm');

After making this change, the application fails to start:

Here is one deviation from the documentation: in order to make the node module oracle-apm that was added to the Node.js installation available to any application – without installing it for every individual application – we need to create a link from the application to the global module:

npm link oracle-apm

in project/application folder. See http://stackoverflow.com/questions/7970793/how-do-i-import-global-modules-in-node-i-get-error-cannot-find-module-module for details.

After creating the link, we can successfully run the application:

The very first time the application is run with the require(‘oracle-apm’) line, the initialization of the application takes place. As part of the initialization, three directories are created under the directory that contains the main entry file of the Node.js application:

One directory is /logs that contains… the log files! These files contain all details about the APM agent and what it does regarding the Node.js application. It is this file that tells us more about the success of running the APM agent and of collecting metrics and sending them to the OMC cloud.

Taking a closer look at the log file, we see two errors. One is expected (says the documenation: http://docs.oracle.com/en/cloud/paas/management-cloud/emcad/verifying-apm-node-js-agent-installation.html ) – […]Error: ENOENT: no such file or directory, open ‘C:\data\jet_on_node\oraclejet-on-nodejs\bin\oracle-apm\identities\AgentIdentity_www.json’. The second should not have been there: ,failure message: unable to get local issuer certificate.

Here comes the second step that is not clearly specified in the documentation: open the file oracle-apm-config.json in directory oracle-apm\config and remove the line for the entry “pathToCertificate”

After deleting the line and saving the file – restart the application. Now, the most pressing warnings and errors have disappeared from the log file. Indications are given that metrics are sent to the OMC environment:

 

The APM Dashboard

In the browser UI exposed by the Oracle Management Cloud Service, we will now find evidence of the agent we have locally configured. The agent has reported in to OMC and has started sending metrics. The first sign of the agent is seen on the Oracle Management Cloud Agents overview page, sub section APM Agents:

The homepage of APM shows a dashboard with various visualizations. All data shown here have been sent to APM by the local agent in Node.JS application, the application running locally on my laptop sitting on my kitchen table in Zoetermeer.

Most squares in the treemap represent JS libraries loaded from Node.js from the browser when the JET application was first accessed. After this initial load, there is no interaction between client (browser) and server (Node.js) and there is no database involved. The analysis we can do with APM for this particular application is extremely limited. However, we can still get a good overview of how many calls were made to the Node.js server and how long each of these took. We can even define alert rules – to have us send an email when the response time is too long.

The next page shows how the server requests – from browser to Node.js server- have been recorded:

On the alerts tab, we can define an alert rule:

Here I specify that an alert should be raised on the warning condition – average Response Time for a specific Page is longer than 100 ms – and on the Critical threshold – the average Response Time for a Page crosses the 400 ms threshold.

After specifying the name, description and conditions, I specify the list of email addresses for all people that should be alerted:

After saving the changes, I sit back and relax – for a very short time, as the alerts start arriving pretty soon:

Opening the email:

and clicking on the Review Details link:

quickly reveals the details belonging to this particular alert. It is quite clear which page has raised the alert. We can investigate which requests took place, from where these happened, and what was done to handle the requests; this will help us find out the requests that took the longest – and why that was the case.

Conclusion

Meaningful conclusions about the application were not yet found. However, we are now in a position to start monitoring the application and find anything worthwhile to be found about the application in the area of non functional user experience and the underlying causes. Hooking up a Node.js application with APM turned out to be fairly straightforward. Next up are perhaps monitoring a Java, ADF or SOA Suite application. And monitoring IT components. And analyzing log files. Stay tuned for more stories.