Enhancing Web Applications with AJAX, XMLHttpRequest, UIX – Report from An AMIS Ten Minute TechTalk

I was kindly requested recently to do a brief – and it was stressed several times that it really had to be brief, not something I am particularly well-known for – presentation on any topic that I considered interesting, relevant and stimulating for our Knowledge Center on Web and Java. My talk was to be the first of many, where all 20 odd members of this Knowlegede Center will do such Ten Minute Talks over the coming months’ meetings. The real challenge for me, apart from that absurd 10 minute limit of course, was not to find a subject – I had dozens. I toyed for example with topics such as: Byte Code generation/manipulation, Oracle Rules, JavaScript and DHTML, ADF Faces/ Java Server Faces, Hibernate, AspectJ and Aspect Oriented Programming, JFreeChart, “XUL, HTC, Flash, XForms and other Client Side technology”, Google Maps, J2EE Design Patterns. I found it very hard to pick just one, primarily because selecting one means de-selecting many others. In the end I decided to demonstrate and discuss the concept of AJAX (Asynchronous JavaScript and XML). Or rather, I discussed a slightly broader definition:

Any method for partially updating the web page displayed in the Browser based on server-interaction without the browser losing focus

I was more concerned with the concepts and potential functionality than the specific technology used. Within this self-defined scope, I discussed three approaches:

1. Frame refresh – Based on multiple frames where one frame is reloaded (and potentially updates the other frames using JavaScript and Client Side DOM Manipulation
2. Oracle UIX – Partial Page Rendering – Based on (hidden) IFRAME and DOM Manipulation (this discusses the built-in features of Oracle UIX and presumably ADF Faces)
3. “regular AJAX”- Based on (hidden) XmlHttpRequestObject and DOM Manipulation

Sort of AJAX, based on frame-refreshing

As an example of the first, rather old-fashioned way of doing AJAX (or at least something which falls under my definition of AJAX), I demonstrated the Repository Object Browser, a tool I developed in the late 1990s using Oracle’s Web PL/SQL Toolkit, largely in PL/SQL using large quantities of JavaScript (no DHTML back then, no DOM manipulation or innerHTML etc.). The ROB (or Oracle Designer Web Assistant as it was called before it was included in the Oracle Designer product in 2003) contains a number of Trees or Navigators. These are all created on the Client, using document.write() statements. Initially, only the top-level nodes for the three are downloaded to the client. Whenever the user expands a tree-node, a (hidden) frame is submitted with a request for the child-nodes of this particular tree-node. When this frame has received the response (onLoad event in the hidden frame), it starts copying the node data to a node-array held in the “top” window and subsequently the tree is completely redrawn – again using document.write. As it turns out, this makes for quite a nice, responsive tree. Compared to the XmlHttpRequest object it feels somewhat clumsy, but it certainly does the job. And since it is the hidden frame that sends the request and gets refreshed, the frame(s) visible and accesible to the user are still available – no loss of focus there.

AJAX in UIX

As discussed in an earlier post – AJAX with UIX – PrimaryClientAction for instantaneous server-based client refresh – Oracle’s UIX has its own AJAX implementation. It uses an IFRAME that is submitted, refreshed and read from – instead of the XMLHttpRequest() object or the ActiveXObject(“Microsoft.XMLHTTPâ€?). This implementation is easy to use, pretty robust – even on somewhat older browsers and quite effective. UIX has the notion of Partial Page Refresh – where only specific components in a page are refreshed through the IFRAME based on the server response to the IFRAME-submit. This submit is triggered by a so-called primaryClientAction. This is a property defined in the UIX page on interactive items such as button and text-inputs. You can link targets – refreshable UIX elements in the UIX page – to a primaryClientAction. Whenever the action occurs, the form with the updated data is sumitted and the target element gets refreshed.

Conceptually, there is very little difference between submitting a request from an IFRAME and creating a request object programmtically and submitting that. The main difference between do it yourself AJAX and UIX is the server side handling of the request: UIX has built in functionality for rendering an appropriate (partial) response – using the same page definitions that also render the main, fullblown pages; when used for a partial render action, the same UIX page suddenly renders just a subset of the nodes, only the ones required for refreshing certain elements. The second big difference lies in the client side handling of the partial response: UIX has JavaScript libraries that know how to refresh the targets of a primaryClientAction from the response received in the IFRAME. Based on element IDs, values are copied from the IFRAME to the main page. This requires no page specific programming whatsoever.

UIX uses this AJAX-like technology (primaryClientActions and associated targets) for these operations:

• expand tree nodes
• sort data in a table (multi-record layout)
• navigate to next or previous data set (in table or multi-record layout)
• perform server-side validation and/or derivation
• select from List of Values
• find value from List of Values based on partial input; this means that for example when the user types in the first one or two letters of the name of a candidate manager, through partial page refresh the application will attempt to complete the name.
• detail disclosure; this is represented through the Hide/Show links: any record in the multi-record layout can be ‘disclosed’. When that happens, through Partial Page Refresh additional fields for this record are retrieved from the server and displayed, just like is done for Employee CLARK and Department ACCOUNTING in the picture below

These UIX specific operations are illustrated in the next picture:

Download

Download the examples of AJAX in UIX: AjaxUixAdfSample.zip Note: this application was generated using JHeadstart; it requires the JHeadstart runtime library (that is included in the zip file). The zip-file furthermore contains a JDeveloper 10.1.2 Application Workspace.

Drawing heavily from the examples I found on the internet, especially those by Tug Grall – Tug Gralls Weblog on AJAX, for example some AJAX examples -, I worked out some examples. You can download all of them below.

The core of each example is a bit of JavaScript that sets up an XmlHttpRequest Object and processes the Response. In each example, the request is slightly different, as is the response and the way the response is to be processed into the webpage. Note that each example starts from a static HTML file.

   // this script is based on code taken from http://www.grallandco.com/blog/archives/ajax/demo-3.html

   var xmlHttpRequest; // global variable, the xmlHttpRequest object that used for AJAX

   /**
   * Create and load the data using the XMLHttpRequest
   */
  function doAjaxRequest(url, object)  // url is the target to send the request to,
                                       // object is an input parameter to the JavaScript function that will process any state change in the request
  {
    // create the object, careful to the MSFT/Other method
    if (window.XMLHttpRequest)
    {
      xmlHttpRequest = new XMLHttpRequest();
    }
    else if (window.ActiveXObject)
    {
      xmlHttpRequest = new ActiveXObject("Microsoft.XMLHTTP");
    }

    // executing the request, passing the targetted object
    xmlHttpRequest.open("GET", url, true); // true indicates ASYNCHRONOUS processing; false would mean SYNCHRONOUS
    xmlHttpRequest.onreadystatechange = function () {processRequestChange(object)};
    xmlHttpRequest.send(null);
  }// doAjaxRequest


  /**
   * Handle the events of the XMLHttpRequest Object
   */
   function processRequestChange(object)
   {
     if (xmlHttpRequest.readyState == 4)
     {
       if(xmlHttpRequest.status == 200)
       {
         processResponse(xmlHttpRequest.responseXML);  // process the response as XML document;
                                                       // alternatively, process xmlHttpRequest.responseText, interpreting the response as plain text or HTML

       }
       else
       {
         alert("Error loading pagen"+ xmlHttpRequest.status +":"+ xmlHttpRequest.statusText);
       }
       // end of the request, change the status zone
       document.getElementById("statusZone").innerHTML = ""
     }
     else
     {
        // Indicates that the client is *busy*
        document.getElementById("statusZone").innerHTML = "<b style='color:red' >Loading...</b>"
     }
   }//processRequestChange

This code assumes that the document contains an element with id statusZone, probably a DIV element, that is used to display messages; for example:

<div  id="statusZone" style="position: absolute; top: 0px; top: 0px; left: 0px; right: 0px; z-index:1;" />

. If that element does not exist, you can simply remove the lines starting with document.getElementById("statusZone").

A typical initiation of an AJAX request would be a call like:doAjaxRequest("http://www.somehost.somedomain/whatever", document.getElementById("ajaxTargetObject").

AJAX – retrieving XML documents, used to populate Select Lists

A typical example of using AJAX is the following: a user can select a value from a Select List, for example Country, Car Make, Department. When he has done so, he can select a value from a second, associated Select List, for example City, Car Type or Employee. Ideally, the values in the second list is restricted based on the selection made in the first list; after having chosen United Kingdom as Country, you would not want to find Cities like Amsterdam, Berlin, New York and Nairobi in the Cities List. Besided, if all possible City, Car Type or Employee values for the second list are loaded along with the page, it takes much longer to load the page and make it available to the user. If the values for the secondary list are only populated when the selection is made in the first list, the initial page load is much faster. The refresh of the second list can be done by a full page refresh, but it would be so much nicer if it happens ‘on the fly’ – asynchronous and instantaneously. Enter AJAX.

In our example, we have a SELECT with a list of Departments. These departments are loaded using AJAX when the HTML document is loaded in the browser. The relevant HTML:

<body onLoad="loadDepartmentData();" class="content" style="text-align:left;margin: 20px 20px 20px 20px;">

  <h1>AJAX with XMLHttpRequest Demonstration - Dependent Lists</h1>

  <p>
  This demonstration shows how ...
<form method="post" action="#" name="demo">
<table>
<tr>
<td>
<p>Department:</p>
</td>
<td>
<select name="dept" id="dept" onChange="loadEmployee();"  class="selectText">
</select>
</td>
</tr>
<tr>
<td>
<p>Employee:</p>
</td><td>

<select name="emp" id="emp" class="selectText" >
</select>
</td>
</tr>
</table>
...

The JavaScript functions used for populating the Department list:

  /**
   * Load the department data in the select object
   * (Could be done in a generic manner depending of the XML...)
   */
  function loadDepartmentData()
  {
    var target = document.getElementById("dept");
    loadXmlData("dept.xml",target);
  }
  <script>

  // this page is taken from http://www.grallandco.com/blog/archives/ajax/demo-3.html
  var xmlHttpRequest;

  /**
   * Create and load the data using the XMLHttpRequest
   */
  function loadXmlData(url, object)
  {
    // create the object, careful to the MSFT/Other method
    if (window.XMLHttpRequest)
    {
      xmlHttpRequest = new XMLHttpRequest();
    }
    else if (window.ActiveXObject)
    {
      xmlHttpRequest = new ActiveXObject("Microsoft.XMLHTTP");
    }

    // executing the request, passing the targetted object
    xmlHttpRequest.open("GET", url, true);
    xmlHttpRequest.onreadystatechange = function () {processRequestChange(object)};
    xmlHttpRequest.send(null);
  }

  /**
   * Handle the events of the XMLHttpRequest Object
   */
  function processRequestChange(object)
  {
    if (xmlHttpRequest.readyState == 4)
    {
       if(xmlHttpRequest.status == 200)
      {
        if (object.id == "emp" )
        {
           copyEmployeeData();
        }
        else if (object.id == "dept" )
         {
           copyDepartmentData();
        }

      }
       else
      {
         alert("Error loading pagen"+ xmlHttpRequest.status +":"+ xmlHttpRequest.statusText);
      }

      // end of the request, change the status zone
      document.getElementById("statusZone").innerHTML = ""
    }
    else
    {
      // Indicates that the client is *busy*
      document.getElementById("statusZone").innerHTML = "<b style='color:red' >Loading...</b>"
    }
  }

  /**
   * Populate the list with the data from the request
   * (Could be done in a generic manner depending of the XML...)
   */
  function copyDepartmentData()
  {
    var list = document.getElementById("dept");
    clearList(list);
    addElementToList(list, "--", "Choose a Department" );
    var items = xmlHttpRequest.responseXML.getElementsByTagName("ROW");
    for (var i=0; i<items.length; i++)
    {
      var node = items[i];
      var deptno = node.getElementsByTagName("DEPTNO")[0].firstChild.nodeValue;
      var dname = node.getElementsByTagName("DNAME")[0].firstChild.nodeValue;
      var loc = node.getElementsByTagName("LOC")[0].firstChild.nodeValue;
      addElementToList(list, deptno, dname  );
    }
  }
  /**
   * remove the content of te list
   */
  function clearList(list)
  {
    while (list.length > 0)
    {
      list.remove(0);
    }
  }

  /**
   * Add a new element to a selection list
   */
  function addElementToList(list, value, label)
  {
    var option = document.createElement("option");
    option.value = value;
    var labelNode = document.createTextNode(label);
    option.appendChild(labelNode   );
    list.appendChild(option);
  }

  </script>

Here is the browser shown after the page was loaded and the list of Departments was populated using XML.

When a Department is selected, the onChange trigger fires and the loadEmployee() function is invoked. This function is shown here:

    /**
   * Load the Employee data in the select object
   * (Could be done in a generic manner depending of the XML...)
   */
  function loadEmployee()
  {
    var target = document.getElementById("emp");
    var deptList = document.getElementById("dept");
    clearList(target);
    if (deptList.value==50)
    {
      loadXmlData('ajaxgetemployees?dept=50',target ); // ajaxgetemployees is a Servlet that returns an XML Document
    }
    else if (deptList.value==20)
    { // invoke a JSP that uses JSTL (SQL and XML) to retrieve Employee details from the database and return an XML document
      loadXmlData('DeptStaffXML.jsp?deptno=20',target );
    }
    else { // retrieve employees from a static XML file on the web server (emp_10.xml, emp_30.xml)
    var file = "emp_"+ deptList .value +".xml"
    loadXmlData(file,target );
    }
  }

/**
   * Populate the list with the data from the request
   * (Could be done in a generic manner depending of the XML...)
   */
  function copyEmployeeData()
  {
  	var list = document.getElementById("emp");
  	var items = xmlHttpRequest.responseXML.getElementsByTagName("ROW");
  	clearList(list);
    if (items.length > 0)
    {
      for (var i=0; i<items.length; i++)
      {
    		var node = items[i];
        var empno = node.getElementsByTagName("EMPNO")[0].firstChild.nodeValue;
        var ename = node.getElementsByTagName("ENAME")[0].firstChild.nodeValue;
        addElementToList(list, empno, ename  );
      }
    }
    else
    {
     alert("No Employee for this department");
    }
  }

Three different methods are used here to produce the XML Document with Employee details: a static XML file on the web server, a Servlet that returns an XML Document and a JSP that accesses the database using JSTL SQL and also returns an XML Document. The Servlet is coded as follows:

package nl.amis.ajax.view;
import javax.servlet.*;
import javax.servlet.http.*;
import java.io.PrintWriter;
import java.io.IOException;

public class AjaxGetEmployees extends HttpServlet  {
  private static final String CONTENT_TYPE = "text/xml; charset=windows-1252";
  private static final String DOC_TYPE;

  public void init(ServletConfig config) throws ServletException {
    super.init(config);
  }

  public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    try {
      Thread.currentThread().sleep(3000); // pause this servlet's (thread) execution for 3 seconds
    }
    catch (InterruptedException e) {

    }

    String deptno = "";
    try {
      deptno = request.getParameter("dept");
    } catch(Exception e) {
      e.printStackTrace();
    }
    response.setContentType(CONTENT_TYPE);
    PrintWriter out = response.getWriter();
    if (DOC_TYPE != null) {
      out.println(DOC_TYPE);
    }
    out.println("<EMP><ROW><EMPNO>100</EMPNO><ENAME>JELLEMA</ENAME></ROW><ROW><EMPNO>200</EMPNO><ENAME>TOBIAS</ENAME></ROW></EMP>");
    out.close();
  }
}

The JSP that used to return Employees from the database makes use of JSTL, specially the SQL library next to the Core library. The DataSource is defined directly in the JSP – better would be to set it up in the Application Server. Note that in order to use the JSTL libraries (in application servers before J2EE 1.4) you need to specify the Tag Libraries in the web.xml file of the application server (see the download file for all code and configuration files). The JSP is coded like this:

<%@ page contentType="text/xml" %>
<%@ taglib uri="/WEB-INF/c.tld" prefix="c" %>
<%@ taglib uri="/WEB-INF/sql.tld" prefix="sql" %>

   <sql:setDataSource user="scott" password="tiger"
                       driver="oracle.jdbc.driver.OracleDriver"
                       url="jdbc:oracle:thin:@localhost:1521:ORCL" var="ds"/>

    <sql:query var="emps" dataSource="${ds}" sql="select ename, empno  from emp where deptno = ?">
      <sql:param value="${param.deptno}"/>
    </sql:query>
<EMP>
  <c:forEach var="row" items="${emps.rowsByIndex}">
<ROW><EMPNO><c:out value="${row[1]}"/></EMPNO><ENAME><c:out value="${row[0]}"/></ENAME></ROW>
</c:forEach>
</EMP>

AJAX – retrieving entire HTML Fragments into a web-page

There are many ways in which to make use of AJAX and a background, asynchronous server request and response. One of the things we can do is have the server send an HTML fragment, for example from a static file, a JSP or Servlet, and paste it somewhere in the requesting page. We can easily include some sort of container object – typically a DIV element – in our webpage and have it absorb and display the server response.

In the next picture we see an example. The page contains three buttons. Each one is linked to a Department. The yellow rectangle is a DIV whose background-color is set to yellow. This is the container element in which the HTML fragment resulting from an AJAX call is published.

When the first button is pressed, the following AJAX call is made: httpRequest.open("GET", "AjaxDivContents.jsp?deptno="+deptno, true); where deptno is a parameter passed in to the JavaScript function. The JSP AjaxDivContents.jsp uses JSTL SQL to retrieve the Employees from the Database and writes an HTML table from those Employee records. This HTML is returned and pasted into the DIV, using the innerHTML property:

          var contentViewer = document.getElementById("contentViewer");
          contentViewer.innerHTML = httpRequest.responseText;

The result looks like this:

Of course when buttons are pressed for other departments, the process is repeated with a different deptno value. Note that the HTML page is loaded once, with the buttons, the JavaScript and the DIV contentViewer element. When a button is pressed, the server is asked – asynchronously – to return a bit of HTML. That HTML is put inside the DIV – while the rest of page is not changed! Only the DIV contents is refreshed.The entire process is illustrated in the next picture:

Initially I had some qualms about using the innerHTML property. I had the impression that FireFox would not support it – but it does. And I had the idea I had to make use of W3C DOM methods, since it would be ‘better’ in some way. However, using cloneNode() or importNode() on the XmlHttpRequest.responseXML() did not give me true HTML elements for some reason – the text was displayed (node values), but not their HTML characteristics.

  tableNode = httpRequest.responseXML.getElementsByTagName('table')[0]; // find the first table element in the XML response
  contentViewer.appendChild(tableNode.cloneNode(true)); // apparently only FireFox

Interestingly enough, this post on a forum demonstrates how using innerHTML to set a piece of fragment, also adds real elements (nodes) into the Document tree. That is, after using innerHTML, you cannot discern between nodes in the Document tree added by createNode() and by innerHTML(). I did not know about performance etc. Then I came across this article: Benchmark – W3C DOM vs. innerHTML. It clearly indicates that using innerHTML is usually much faster than achieving the same effect using W3C DOM methods.

This put together really converted me: innerHTML is fine! Actually, it is pretty cool.

The JSP that returns the HTML fragment looks like this. Note that it does not include HTML and BODY tags, it is just a fragment:

<%@ page contentType="text/html" %>
<%@ taglib uri="/WEB-INF/c.tld" prefix="c" %>
<%@ taglib uri="/WEB-INF/sql.tld" prefix="sql" %>
   <sql:setDataSource user="scott" password="tiger"
                       driver="oracle.jdbc.driver.OracleDriver"
                       url="jdbc:oracle:thin:@localhost:1521:ORCL" var="ds"/>

    <sql:query var="emps" dataSource="${ds}" sql="select ename, empno  from emp where deptno = ?">
      <sql:param value="${param.deptno}"/>
    </sql:query>
<h3>Here are the employees of department <c:out value="${param.deptno}"/></h3>
<table id="tbl"><tr><th>Empno</th><th>Name</th></tr><c:forEach var="row" items="${emps.rowsByIndex}">
<tr><td><c:out value="${row[1]}"/></td><td><c:out value="${row[0]}"/></td></tr>
</c:forEach>
</table>

Needless to say that the DataSource really should be defined at Servlet Container level and not hardcoded in this JSP.

Download

Download the examples of regular AJAX – using static XML, JSP/JSTL and Servlet on the Server Side: AjaxExample.zip. Again, this a JDeveloper 10.1.2 Application Workspace. However, the HTML, JSP/JSTL and Servlet Code is not tied to JDeveloper in any way and all code can be run with any Servlet Container or Java IDE.

Googling on AJAX results in a wealth of hits. There are a few that I found very helpful, so these I will mention:
AJAX in WikiPedia
Apache Direct Web Remoting (DWR), an introduction
Fifty Four Eleven – a huge collection of references to AJAX examples and articles
http://www.papermountain.org/demos/live/# – a very good demo of some straightforward AJAX applications
AJAX with BabySteps – a very useful tutorial
JSON – JavaScript Object Notation – a library for data interchange: exchanging Java Objects through JavaScript for example
Very Dynamic Web Interfaces by Drew McLellan February 09, 2005 (article on O’Reilly XML.com)
Tug Gralls Weblog on AJAX – which provided some very good demos