How to flatten an existing JavaScript codebase

Since my previous article on async/await ended up getting popular I thought I'd follow up with more focused guide on flattening your JavaScript. We started this process on the Expedited Security codebase a couple of weeks ago and while it was mostly straightforward, there were a couple of surprises.

This guide applies to both frontend JavaScript and node. Remember async/await is supported in:

• All current browsers
• Node since version 18 - current LTS versions is better though since it comes with util.promisify(), allowing the node stdlib to return Promises and this be used with await.

1. Install and configure eslint

jslint doesn't handle await yet. So install it's nerdier cousin, eslint. Your .eslintrc.json should contain:

"parserOptions": {
    "ecmaVersion": 2017
},

2. For node: handle uncaught rejections

If a promise rejects - ie, an error occurs - and you don't catch it, you'll see:

(node:14104) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: (some error here)
(node:14104) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

You won't get a traceback to find the offending line though. So add this to the first file in your app (eg, bin/www for an Express app):

process.on('unhandledRejection', function(reason, promise) {
    console.log(promise);
});

To get a full traceback like any other error.

3. Pick a small, isolated set of code

Expedited Security uses various REST APIs for government company registrars, independent information sources, whois, DNS, Microsoft Cognitive Services and a bunch of our own logic. Each has a bunch of integration tests and mocks.

API clients are perfect for beginning your code flattening as they're:

• do a bunch of async work, eg. HTTP requests
• self-contained - query in, results and errors out.

4. Choose your weapons

To use await, you'll need your existing libraries to support Promises. The most common async task in an API client is HTTP requests, and you have a few options here:

• superagent runs in both the browser and the server and already supports promises. Like a lot of node APIs it supports both callbacks and promises. Just omit the callback and it will return a Promise instead.
• If you're using jQuery for your AJAX, jQuery version 3 or newer has working support. Older jQuery has slightly busted promises, though, so do upgrade if you're on an older release.
• You could also use the fetch API if you want to write your own query string encoding, MIME type handling, etc. but you probably don't want to. superagent does those things and you'll end up writing less code by using it instead.

node 8 includes util.promisify() that can be used to wrap older node libraries. So you can make a Promisified version of fs.readFile with:

const readFile = util.promisify(fs.readFile);

Then use that wrapped function - just await it and omit the callback.

const fileContents = await readFile('somefile.txt');

5. Modify your code to use async/await

The first step is simple:

superagent.get('/api/v1/some-resource', function(err, result){
    if ( err ) {
        // Handle err
        return
    }
    // Handle result...
});

Becomes:

var result = await superagent.get('/api/v1/some-resource');

You'll also need to add the async function to the parent scope (eslint will remind you about this with unexpected token).

You don't have to handle errors if you don't want to, but this is easily done. Since APIs break a lot, it's best that we do:

try {
    var result = await superagent.get('/api/v1/some-resource');
} catch(err) {
    log(`Oh no an error occured ${err.message}`)
}

6. Work out async/await equivalents of common asynchronous coding patterns.

This often means turning examples using .then() syntax into async/await versions. Here's some handy examples:

Doing a bunch of things in parallel and then doing something with the combined result:

var log = console.log;
var getBanana = function() {
    return new Promise(function(resolve, reject) {
        setTimeout(function(){
            resolve('banana')
        }, 2000);
    });
}
var getGrape = function() {
    return new Promise(function(resolve, reject) {
        setTimeout(function(){
            resolve('grape')
        }, 3000);
    });
}
var start = async function(){
    var result = await Promise.all([getBanana(), getGrape(), 'apple'])
    console.log('All done', result)
}
start()

Will log ['banana', 'grape', 'apple']

Handling _.debounce() with await

var log = console.log.bind(console),
    _ = require('lodash')
var log = console.log.bind(console);
var resultsAfterDelay = new Promise(function(resolve) {
    _.debounce(function(){
        resolve('foo');
    }, 100)();
});
var start = async function(){
    log(await resultsAfterDelay)
}
start();

Modify your existing code and unit tests to call the new function signatures.

await tends to make parents code flatter too. For example:

• You had a function that did a GET request, then ran a callback with the result
• Your function now uses = await to GET the resource
• The function can now simply return a result
• Code that calls that function can now use = await and omit the callback.

You'll notice await tends to creep down the stack:

A lot of editors and command line tools can properly find all instances of a function being called, but the most basic tool you can use it git grep.

For unit tests - we use mocha - changing from callbacks to await means you can simply:

• Remove the done callback
• Remove explicit error handling - thrown Errors will be caught by Mocha.

Essentially, async unit testing with await looks like syncronous unit testing.

So your API client is nice and flat. But the thing that's calling it via await is still using callbacks. That's a reasonable intermediate step - these things take time.

We've been approaching flattening our code Joel style: before we work on new features, we'll clean any remaining callbacks in the code first. It's been going well so far, with the more popular code paths being largely flattened.