How to Compose Node.js Promises with Q

In the last article on promises, we focused on the Promises/A+ specification and Q implementation of promises. We learned the benefits of using promises over raw callbacks. Now, we’re going to take it up a notch and focus on promise composition, which we’ll define as functional programming and asynchronous control flow using promises.

More about chaining

The then method allows us to chain promises (see 3.2.6 in Promises/A+ spec). The value returned from a chain of promises is itself a promise. This returned promise will be resolved to the value returned from the last onFulfilled or onRejected handlers in the chain. Let’s look at some examples:

var chainPromise = first().then(second).then(third).then(fourth)
chainPromise.then(console.log, console.error)

Here, chainPromise could either be:

1. Fulfilled with the return value of fourth since it was the last onFulfilled handler in the chain or
2. Rejected at any point of the chain since there are no onRejected handlers

The same is true for nesting; this code will produce the same behavior as the above example:

var nestedPromise = first().then(function (val1) {
  return second(val1).then(function (val2) {
    return third(val2).then(function (val3) {
      return fourth(val3)
    }
  })
})
nestedPromise.then(console.log, console.error)

Armed with this knowledge, we can create a recursive chain that calls a function forever until an error occurs (like async.forever):

function forever (fn) {
  return fn().then(function () {
    return forever(fn)  // re-execute if successful
  })
}
// console.error only ever called if an error occurs
forever(doThis).then(undefined, console.error)

Won’t this blow the stack? Unfortunately, JavaScript does not have proper tail call support yet. However, it won’t affect this recursive call because Promises/A+ requires the onFulfilled and onRejected handlers to be called on a future turn in the event loop after the stack unwinds (3.2.4 in Promises/A+).

Starting chains and grouping promises

In addition to the functional programming concepts we enjoy in synchronous programming, promise libraries like Q provide tools to aid in composition. We’ll focus on two of them: Q() and all.

Q(value) helps us start promise chains. If no value is provided, a promise is returned fulfilled to undefined. We’ll call this an “empty” promise. If a value is provided, a promise is returned fulfilled to that value:

Q('monkeys').then(console.log) // will log 'monkeys'

Q() also converts promises from other libraries, but we won’t be using that here.

The second tool is all which has static (Q.all) and instance (promise.all) methods. all takes an array of promises and returns a new promise, which we’ll call a “group” promise. The group promise will either be resolved when all the promises have been fulfilled or rejected when any have been rejected.

all is helpful for grouping the fulfillment values from multiple promises, regardless if the execution is done in series or parallel.

The static (Q.all) method looks like this:

var groupPromise = Q.all([ doThis(), doThat() ])
groupPromise.then(function (results) { }, console.error)

all maintains the ordering of the results array, so the result of doThis() would be index `` and so on. If either doThis() or doThat() had an error, groupPromise would be rejected and we’d log it with console.error.

The instance (promise.all) method is typically used in chains and looks like this:

Q()
  .then(function () {
    return [ doThis(), doThat() ] // return a list
  })
  .all()
  .then(function (results) { }, console.error)

Note that promise.all has the same function signature as then, so we could have just said:

.all(function (results) { }, console.error)

Working with collections

Let’s look at iterating through collections of data that require asynchronous action per element. First, let’s mimic async.map using promises:

function map (arr, iterator) {
  // execute the func for each element in the array and collect the results
  var promises = arr.map(function (el) { return iterator(el) })
  return Q.all(promises) // return the group promise
}

We could then utilize this function as such:

// turn fs.stat into a promise-returning function
var fs_stat = Q.denodify(fs.stat)
map(['list', 'of', 'files'], fs_stat).then(console.log, console.error)

The beauty of this approach is that any function will work, not just promise-returning ones. For example, let’s say we wanted to stat only the files we do not already have in our cache:

var cache = Object.create(null) // create empty object
function statCache (file) {
   // return the cached value if exists
  if (cache[file]) return cache[file]

  // generate a promise for the stat call
  var promise = fs_stat(file)
  // if that promise is fulfilled, cache it!
  promise.then(function (stat) { cache[file] = stat })

  return promise // return the promise
}
map(['list', 'of', 'files'], statCache).then(console.log, console.error)

Here, statCache returns a value or a promise. Regardless of what’s returned, we can group it and provide the results with all. Sweet!

How can this work? The all method also takes in values as arguments which internally it tranforms into a promises fulfilled to those values.

However, there is a problem with our map function. What if an exception is thrown in statCache? Right now, the exception wouldn’t be caught since it isn’t in a promise chain. Here is where Q() comes in:

function map (arr, func) {
  return Q().then(function () {
    // inside a `then`, exceptions will be handled in next onRejected
    return arr.map(function (el) { return func(el) })
  }).all() // return group promise
}

So we talked about iterating through a collection with promises in parallel, but what about iterations in series (like async.mapSeries)? Here is one approach (thanks @domenic):

function mapSeries (arr, iterator) {
  // create a empty promise to start our series (so we can use `then`)
  var currentPromise = Q()
  var promises = arr.map(function (el) {
    return currentPromise = currentPromise.then(function () {
      // execute the next function after the previous has resolved successfully
      return iterator(el)
    })
  })
  // group the results and return the group promise
  return Q.all(promises)
}

In the above example, we used all to group operations done in series and Q() to start our promise chain. Each time through arr.map, we built a larger chain and returned a promise for that point in the chain until we reached the end of the array. If we unraveled this code, it would look something like this:

var promises = []
var series1 = Q().then(first)
promises.push(series1)
var series2 = series1.then(second)
promises.push(series2)
var series3 = series2.then(third)
promises.push(series3)
// ... etc
Q.all(promises)

When structured this way, we maintain the order (seconds won’t be called until firsts are done) and gain grouping (the last

promise fulfillment value in each chain is grouped).

Going further with promises

For more examples of chaining and grouping promises, this gist implements most of the async API. However, your best grasp of these concepts will come by playing around with them. Here are some ideas:

1. Try implementing map or mapSeries another way
2. Implement the async API using another promise library
3. Write a network server (web or otherwise) using promises — for inspiration, check out mach

And from the previous article:

1. Wrap some basic Node workflows converting callbacks into promises
2. Rewrite one of the async methods into one that uses promises
3. Write something recursively using promises (a directory tree might be a good start)
4. Write a passing Promise A+ implementation. Here is my crude one.

##Use StrongOps to Monitor Node Apps

Ready to start monitoring event loops, manage Node clusters and chase down memory leaks? We’ve made it easy to get started with StrongOps either locally or on your favorite cloud, with a simple npm install.

What’s next?