Sequential Asynchronous calls in Node.JS – using callbacks, async and ES6 Promises

One of the challenges with programming in JavaScript (ECMA Script) in general and Node.JS in particular is having to deal with asynchronous operations. Whenever a call is made to a function that will handle the request asynchronously, care has to be taken to be prepared to receive the result from the function in an asynchronous fashion. Additionally, we have to ensure that the program flow does not continue prematurely – only those steps that can be performed without the result from the function call can proceed. Orchestrating multiple asynchronous – some of them sequential or chained and others possibly in parallel – and gathering the results from those calls in the proper way is not trivial.

Traditionally, we used callback functions to program the asynchronous interaction: the caller passed a reference to a function to the asynchronous operation and when done with the asynchronous operation, the called function would invoke this callback function to hand it the outcome. The call(ed)back function would then take over and continue flow of the program. A simple example of a callback function is seen whenever an action is scheduled for execution using setTimeout():

setTimeout(function () {
  console.log("Now I am doing my thing ");
}, 1000);

or perhaps more explicitly:

function cb() {
  console.log("Now I am doing my thing ");
}

setTimeout(cb, 1000);

Chain of Asynchronous Actions

With multiple mutually dependent (chained) calls, using callback functions results in nested program logic that quickly becomes hard to read, debug and maintain. An example is shown here:

Function readElementFromJsonFile does what its name says: it reads the value of a specific element from the file specified in the input parameter. It does so asynchronously and it will call the callback function to return the result when it has been obtained. Using this function, we are after the final value. Starting with file step1.json, we read the name of the nextfile element which indicates the next file to read, in this case step2.json. This file in turn indicates that nextStep.json should be inspected and so on. Clearly we have a case of a chain of asynchronous actions where each action’s output provides the input for the next action.

In classic callback oriented JavaScript, the code for the chain of calls looks like this – the nested structure we have come to expect from using callback functions to handle asynchronous situations:

// the classic approach with nested callbacks
var fs = require('fs');
var step1 = "/step1.json";

function readElementFromJsonFile(fileName, elementToRead, cb) {
    var elementToRetrieve = 'nextfile';
    if (elementToRead) {
        elementToRetrieve = elementToRead;
    }
    console.log('file to read from ' + fileName);
    fs.readFile(__dirname + '/' + fileName, "utf8", function (err, data) {
        var element = "";
        if (err) return cb(err);
        try {
            element = JSON.parse(data)[elementToRetrieve];
        } catch (e) {
            return cb(e);
        }
        console.log('value of element read = ' + element);
        cb(null, element);
    });
}//readElementFromJsonFile

readElementFromJsonFile(step1, null, function (err, data) {
    if (err) return err;
    readElementFromJsonFile(data, null, function (err, data) {
        if (err) return err;
        readElementFromJsonFile(data, null, function (err, data) {
            if (err) return err;
            readElementFromJsonFile(data, null, function (err, data) {
                if (err) return err;
                readElementFromJsonFile(data, 'actualValue', function (err, data) {
                    if (err) return err;
                    console.log("Final value = " + data);
                });
            });
        });
    });
});

The arrival of the Promise in ES6 – a native language mechanism that is therefore available in recent versions of Node.JS – makes things a little bit different and more organized, readable and maintainable. The function readElementFromJsonFile() will now return a Promise – a placeholder for the eventual result of the asynchronous operation. Even though the result will be provided through the Promise object at a later moment, we can program as if the Promise represents that result right now – and we can anticipate in our code at what to do when the function delivers on its Promise (by calling the built in function resolve inside the Promise).

The result of the resolution of a Promise is a value – in the case of function readElementFromJsonFile it is the value read from the file. The then() operation that is executed when the Promise is resolved with that value, calls the function that it was given as a parameter. The result (resolution outcome) of the Promise is passes as input into this function. In the code sample below we see how readElementFromJsonFile(parameters).then(readElementFromJsonFile) is used. This means: when the Promise returned from the first call to the function is resolved, then call the function again, this time using the outcome of the first call as input to the second call. With the fourth then this is a little bit more explicit: since in the final call to the function readElementFromJsonFile we need to pass not just the outcome from the previous call to the function as an input parameter but also the name of the element to read from the file. Therefore we use an anonymous function that takes the resolution result as input and makes the call to the function with the additional parameter. Something similar happens with the final then – where the result from the previous call is simply printed to the output.

The code for our example of subsequently and asynchronously reading the files becomes:

var fs = require('fs');
var step1 = "step1.json";

function readElementFromJsonFile(fileName, elementToRead) {
    return new Promise((resolve, reject) => {
        var elementToRetrieve = 'nextfile';
        if (elementToRead) {
            elementToRetrieve = elementToRead;
        }
        console.log('file to read from ' + fileName);
        fs.readFile(__dirname + '/' + fileName, "utf8", function (err, data) {
            var element = "";
            if (err) return reject(err);
            try {
                element = JSON.parse(data)[elementToRetrieve];
            } catch (e) {
                reject(e);
            }
            console.log('element read = ' + element);
            resolve(element);
        });
    })// promise
}

readElementFromJsonFile(step1)
    .then(readElementFromJsonFile)
    .then(readElementFromJsonFile)
    .then(readElementFromJsonFile)
    .then(function (filename) { return readElementFromJsonFile(filename, 'actualValue') })
    .then(function (value) { console.log('Value read after processing five files = ' + value); })

Scheduled Actions as Promise or how to Promisify setTimeout

The setTimeout() built in expects a call back function. It does not currently return a Promise. Something like:

setTimeout(1000).then(myFunc)

would be nice but does not exist.

This entry on Stackoverflow has a nice solution for working with setTimeout Promise style:

function delay(t) {
   return new Promise(function(resolve) { 
       setTimeout(resolve, t)
   });
}

function myFunc() {
    console.log('At last I can work my magic!');
}

delay(1000).then(myFunc);