'You can't use Brotli for dynamic content'

Brotli is a new standard that provides better compression than gzip - it's now supported in all 4 major browsers.

Checking the top 1000 URLs on the internet, brotli performance is:

• 14% smaller than gzip for JavaScript
• 21% smaller than gzip for HTML
• 17% smaller than gzip for CSS

The first one really excites us. Modern websites in particular often have large JavaScript bundles - the front page of CertSimple is 242 K gzipped, and would be 1.1MB uncompressed!

Which means it's really easy to take this - a 242KB, gzipped, minified JS bundle:

And turn it into this - a 201KB brotli-compressed JS bundle:

While not as big a help to site performance as HTTP/2's awesome multiplexing, it's still a decent speed benefit for only a small amount of work. And it's nicely compatible: visitors with an out of date browser, can still get regular old gzipped resources.

Getting brotli running isn't hard, but there's a couple of places where you can go amiss, especially if you follow most of the current 'Brotli nginx' articles.

Mainly because they contain one big fat lie: that Brotli is slower than gzip to compress, so you can't use it for dynamic content. This is a bunch of horseshit.

So here's a list of important considerations when adding brotli to your site.

1. Brotli can compress faster than gzip and still produce smaller files

You may have read that Brotli is 'slower than gzip'. The first time this article was posted on Reddit, someone responded:

Brotli is slow for compression. I enabled this on a static asset management tool a while ago and it took several times longer to generate compressed files than gzip. I wondered if this was a function of tooling but it looks like its an artifact of the compression algorithm.

This comment was upmodded, since anyone who uses the expression 'artifact of the compression algorithm' clearly knows what they're talking about, and people on Reddit don't actually read articles they comment on. There's only one problem: it's wrong.

It's not unique though - you'll hear that in a lot of Brotli tutorials too. And yes, if you take a file, and run:

time brotli --input input.js --output input.js.br
time gzip input.js

The first command will take much longer than the second. But that can be misleading: both gzip and brotli have variable levels of compression, and the brotli command is turned up to max by default.

To summarize Akamai's study on Brotli performance...

You can take advantage of that in your Brotli deployment! Eg, for nginx:

• For static assets, we'll compress the hell out of them with pre-compressed .br files using 11 - since we can serve these immediately for every request.
• For dynamic content, we'll use 4, which still produces smaller responses but takes less time to compress than gzip or brotli on a higher setting. This way browsers only have to wait a short time for us to compress the responses.

Brotli has modules for each:

• brotli_filter compresses responses on-the-fly, just like gzip
• brotli_static serves pre-created .br versions of files, meaning you have to create the files with the brotli command as part of your deploy.

To load the modules into nginx, add the load_module directive in the top‑level block of your nginx.conf:

# Compress responses on-the-fly.
load_module modules/ngx_http_brotli_filter_module.so;
# Serve pre-compressed files.
# Both modules could be used separately
load_module modules/ngx_http_brotli_static_module.so;

You can use each module seperately if you wish.

Then in the http block, add:

brotli on;
brotli_comp_level 4;
brotli_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;
brotli_static on;

Make a script to go into your app's 'public' directory, finding files ending in 'css', 'js', 'json' or 'svg', and using the 'brotli' command to create files with the same name, using 11 compression, and a .br extension. The 'force' isn't as scary as it sounds: it just overwrites existing files.

#!/usr/bin/env bash
OLDIFS=$IFS
IFS=$'\n'
for FILE in $(find public -type f -iname '*.css' -o -iname '*.js' -o -iname '*.svg' -o -iname '*.json'); do
    echo -n "Compressing ${FILE}..."
    brotli --input ${FILE} --force --output ${FILE}.br;
    echo "done."
done
IFS=$OLDIFS

Results will look like:

Compressing public/images/blog/infinite.svg...done.
Compressing public/images/blog/certificate-chain-root.svg...done.
Compressing public/images/blog/trust-seals-are-rubbish.svg...done.
Compressing public/images/blog/carwash-results.svg...done.
Compressing public/images/blog/twitter.svg...done.
Compressing public/images/blog/cross-signing.svg...done.
Compressing public/images/blog/logo-copy.svg...done.
Compressing public/images/blog/why-do-i-need-ssl.svg...done.

Then restart nginx:

sudo systemctl restart nginx

2. nginx might not be the best place to set up Brotli

OK, great, that's nginx covered. But most people build their JS bundles on a their app server with tools like Browserify or Webpack. Unless nginx is running on the same machine, brotli_static won't help. To repeat:

Because brotli_static, like gzip_static before it, is only for files served via nginx - it will only detect .br files when served via the filesystem, not via proxy_pass used on a typical nginx load balancer. Quoting nginx core engineer Maxim Dounin:

_static is only expected to work when nginx is about to return regular files.

So how do you get brotli working on your environment? You could add a second layer of nginx sitting on each app server, serving your JS bundles on the same machine, but that's a lot of extra complexity.

Most languages have a native Brotli module - we use node's amazing 'shrink ray'. Shrink-ray:

• handles both Brotli and Zopfli (a better version of gzip for old browsers)
• includes Express middleware
• is a drop-in replacement for the older compression module

In short: 😍 Shrink Ray.

If you use Rust: the mysteriously named rust-brotli will help you out.

I'll update this post with Ruby, Python, Elixir, and Go equivalents as we discover them.

In this case, you can disable compression on nginx and let the app servers take care of using both brotli and gzip as necessary. If you're wondering: yes, you can still use nginx for HTTP/2 in this setup.

2. Building brotli from source is unnecessary

If nginx does work for you, the next think you'll see on most brotli guides are instructions to build nginx's brotli modules.

You can save yourself the effort.

For Ubuntu 16.04

Ubuntu 16.04 normally comes with the older 1.10 nginx release, and doesn't include brotli, but we can use cryptofuture's nginx repo to add nginx 1.11 and brotli. Install the updated nginx, the brotli module, and the command used to make .br files:

sudo apt-add-repository -y ppa:hda-me/nginx-stable
sudo apt-get update
sudo apt-get install brotli nginx nginx-module-brotli

Let's check nginx has brotli support:

nginx -V 2>&1 | tr ' '  '\n' | grep brotli

You'll see:

--add-dynamic-module=debian/extra/ngx_brotli

You now have a working brotli setup.

For RHEL / CentOS 7

If you'e more a RHEL/CentOS types, CodeIT has nginx packages with the brotli module built in.

cd /etc/yum.repos.d
wget https://repo.codeit.guru/codeit.mainline.el`rpm -q --qf "%{VERSION}" $(rpm -q --whatprovides redhat-release)`.repo
yum install nginx

Afterwards, you can see the brotli module is included with nginx itself:

nginx -V 2>&1 | tr ' '  '\n' | grep brotli

Will return:

--add-module=/tmp/nginx/ngx_brotli

Boom. nginx with brotli.

3. Brotli isn't for every file type

Many nginx brotli guides include:

brotli_types *;

Binary files like JPEG, PNG, MP4, are already compressed with format-specific compression which outperforms the naive compression of gzip and brotli. There's no point gzipping or brotli-ing a PNG: the PNG is already compressed, and it will get bigger rather than smaller. So use:

brotli_types text/plain text/css application/javascript application/json image/svg+xml application/xml+rss;

To cover HTML (included by default), plaintext, JavaScript, JSON, SVG and RSS.

5. If you are using nginx, disable gzip compression on your backend servers

Using nginx in front of your backend servers? nginx won't recompress responses as brotli if your backend servers are already providing them as gzip. Disable gzip on your backend to use brotli on nginx.

Checking the results

In Chrome, start Developer Tools (Ctrl Shift I) and click the Network tab.

Click on network requests a JS, HTML or SVG file and you should see Accept-Encoding: br in the request headers and Content-Encoding: br in the response.

Is Brotli worth it? Using Brotli we've seen a 17% improvement in one of the largest JS bundle we serve to customers. Despite the some of the surprises above, we think it's a quick win for most websites. Happy Brotling!