The problem I am dealing with: downloading all data that is potentially to be displayed in the client – for example in SELECT LISTS – takes too long to do for every page and even for every session.
There are quite a few ways of saving data on the client during a session across page loads, such as IFRAME and (hidden) text field. However, these fall short when the browser session ends. And we need faster data access for all our sessions.
Possible Solutions:
- Use cookies to store data across sessions
- Store data in JavaScript libraries and relying on client side caching
- Use AJAX to download while the user is doing other things
- Use AJAX to download only what it is actually required
- Use of Interner Explorer Behaviors to store session data on the client
In this article, I briefly and very superficially explore these possible solutions.
Use of cookies
Cookies can only store small amounts of data – about 4K. That means they are not a viable solution. Besides, the contents of the Cookie is sent across with each request. That would be defeating the purpose, as the data would actually be sent across the wire more frequently instead of less frequently.
Use of AJAX
It is certainly an option to use AJAX techniques to perform the load operation after the page has loaded and while the user is doing other stuff. Caching this data for the duration of the session is easy: by creating JavaScript data structures in a ‘persistent’ IFRAME (a frame that is invisible and is not refreshed when the rest of the page is reloaded) we can retain those structures on the client.
Whether we can use AJAX to restrict the volume of data downloaded – only download what is actually required – depends on the required functionality of the application. If we can design the application in such a way that the user does not have to jump right away into a list with 1000+ items but instead can make a preselection – by first picking a category or typing the first few letters – we can use AJAX to implement such partial downloads. They can probably be quick enough as to not use caching.
We depend on the Analysts to decide if and how we can build selection lists in (AJAX-able) steps rather than the whole set at once.
Use of JavaScript Libraries
First of all: data can be stored in JavaScript libraries, for example in functions like:
function add(label, value) {
list.options[ x].value
list.options[ list.options.length] = new Option(label, value);
}
function initData() {
add("First Item in the List", "value1");
add("Second Item in the List","value2");
...
}
Second of all: when JavaScript is not included in the page, but instead in an external library that is referenced from the webpage:
<script src="myfile.js"></script>
it will be cached in the Browser Cache.
Because you have moved the common source into a separate file, the file only has to be downloaded once. After the first download, the file stays in the browser cache, and never has to come over the wire again. That is: until the file is modified on the server. The browser will reload the JavaScript library if it detects that the file on the server is newer. Note: the above description is dependent on the Browser Cache Settings.
The user can specify if and when the cache should be updated with newer files from the server.
Of course if the reference to javascript library in the webpage changes, for example to:
<script src="myfile_version2.js"></script>
it will also be (re)loaded, as it does not yet exist in the browser cache.
DEFER Your Scripts.
<script src="myfile_version2.js" DEFER="true" >
Or just:
<script src="myfile_version2.js" DEFER >
DEFER is a relatively obscure attribute of the script element, but the performance-minded page author can use it to indicate to Internet Explorer 4.0 or later that the script tag contains no immediately executing code that impacts the document before the load event fires. Combined with the SRC attribute, the DEFER attribute can be used on a script tag to indicate to Internet Explorer that the associated code should be downloaded in the background while the rest of the components of the page are downloaded and parsed.
Leverage the HTTP Expires Header
The expires header is part of the HTTP 1.0 specification. When an HTTP server sends a resource such as an HTML page or an image down to a browser, the server has the option of sending this header and an associated time stamp as part of the transaction. Browsers typically store the resource along with the expiry information in a local cache. On subsequent user requests for the same resource, the browser can first compare the current time and the expires time stamp. If the time stamp indicates a time in the future, the browser may simply load the resource from the cache rather than retrieving the resource from the server.
Even when a resource would advertise an expiration date still to come, browsers—including Internet Explorer 4.0—would still perform a conditional GET to determine that the version of the object in the cache was the same as the version on the server. Upon careful analysis, the designers of Internet Explorer determined that this extra round trip was neither optimal nor necessary. For that reason, the behavior of Internet Explorer 5 has been modified in the following way: if the expiry of a cached resource is later than the time of the request, Internet Explorer will load the resource directly from the cache without contacting the server. Sites using the expires header on commonly used but infrequently updated resources will experience lower traffic volumes, and customers using Internet Explorer 5 will see pages render more quickly.
Note: we can set this header only for JavaScript libraries that are dynamically created by a servlet or JSP. I do not believe we can do this for static files, although I may be mistaken.
Scenarios for using JavaScript libraries to cache and refresh
Using JavaScript libraries is clearly a way to have data persisted on the client between sessions. However, there are several strategies for refreshing the data on the client.
1. One static JavaScript library
The easiest approach is probably with a single JavaScript library, as static file on the web server. This file is cached on the clients. Whenever a data element is changed, a new static library is written on the web server and the clients will automatically refresh the file.
Disadvantages: a new library must be created on the web server for every change in the data that is to presented in the client and even for the smallest change, the entire file is refreshed (=-reloaded). If there are frequent changes, this is hardly better than only saving data during a session.
2. Multiple static JavaScript libraries
A way around some of the disadvantages of the solution with a single static JavaScript library is the use of multiple libraries.
We can have a library per data set – meaning we only have to update the libraries that contain changes – which means much lower volume of reloading to the clients.
Another or additional approach can be to have a static library per data-set (or even all data-sets combined) complemented with one shared library that contains all changes with regard to the static libraries. If the data that we want to cache changes by less than say 5 or 10%, we gain 90% download time by having the static library cached all the time while only the much smaller library (or libraries) with changes need to be refreshed on a more regular base. At some point (after six months or a year, depending on the numberof data changes) we can merge all changes into the large library, shrink back the ‘library of changes’ and have the clients take a single download hit for refreshing the big library (or libraries).
3. Multiple static JavaScript libraries complemented by Dynamic “updates�
This last scenario can also be done in a different way: we still have the static library (or libraries) with the base set of cacheable data. However, the library (libraries) with the additional changes are not static files: instead they are servlet calls, and the changes are dynamically created. However, the servlet takes good care to set the http header signifying the last changed date, to prevent the client from reloading the library when there are no changes compared to the last download. The advantage of this approach over the former is that now changes in the data are automatically available to the clients, without the intermediate need for creating a static library with changes.
It is also an option to improve on this through the use of AJAX: the loading and applying of (the very latest) changes can very well be done using AJAX. The JavaScript library provides the bulk of all data – from the cache – while a more refined AJAX mechanism retrieves recent updates from a Servlet or JSP and applies them to the previously loaded data from JavaScript.
Note that these scenario’s all require to some extent the registration of when a record is changed (created, updated or deleted (!)).
Use of Interner Explorer Behaviors
Persistence enables authors to specify an object to persist on the client during the current and later sessions using Dynamic HTML (DHTML) behaviors. Persistence allows Microsoft Internet Explorer 5 and later to retain Web page information, styles, variables, and state. For example, a collapsible list of links within a table of contents can remain expanded to the user’s choice upon leaving and later returning to the page. Or, a search engine query form can retain the last-used search string.Persistence is implemented as a behavior.
See: http://msdn.microsoft.com/library/default.asp?url=/workshop/author/persistence/overview.asp
The userData behavior persists information across sessions by writing to a UserData store. This provides a data structure that is more dynamic and has a greater capacity than cookies. The capacity of the UserData store depends on the security zone of the domain. The following table shows the maximum amount of UserData storage that is available for an individual document and also the total available for an entire domain, based on the security zone.
Security Zone Document Limit (KB) Domain Limit (KB) Local Machine 128 1024 Intranet 512 10240 Trusted Sites 128 1024 Internet 128 1024 Restricted 64 640
The userData behavior persists data across sessions, using one UserData store for each object. The UserData store is persisted in the cache using the save and load methods. Once the UserData store has been saved, it can be reloaded even if Microsoft Internet Explorer has been closed and reopened.
Client Persistence and UserData are IE specific. They provide much greater programmatic control than does the use of cached JavaScript libraries. However, they have a larger dependency on browser settings and there is a limitation in capacity – 128 Kb – that, while much more spacious than the Cookies, may still render it useless.
Resources
http://codinginparadise.org/weblog/2005/08/ajax-tutorial-saving-session-across.html – explains how to save state on the client across pages –during a single browser session; one method is through iFrames, another is using hidden textFields: if text is stored in a textField (hidden or not) inside a form, then any page in the same browser session with that form and field will contain the same data; thus data can be kept on the client side between page reloads.
http://msdn.microsoft.com/library/default.asp?url=/workshop/author/persistence/overview.asp – article on MSDN, introducing the concept of client side persistence, with a number of useful code examples
http://msdn.microsoft.com/library/default.asp?url=/workshop/author/perf/perftips.asp – the Internet, intranets, and extranets carry millions of packets of data. Today, many of those packets contain HTML. The features delivered with Microsoft Internet Explorer 4.0 and later have helped make the Web a compelling space in which to work and to play. The quantity and complexity of pages as well as the number of consumers of those pages has significantly increased the traffic on the Web. For all the merit that the Web brings to application developers, it introduces a host of problems. Among these problems are:
* Delivering content across the wire.
* Once delivered, getting that content to render quickly.
This article presents some tips on how you can get the most performance out of your pages.
http://www.eggheadcafe.com/articles/20010615.asp – UserData Intrinsic IE Persisitence Behavior Client – Side Data Store By Peter A. Bromberg, Ph.D.
