An Adventure in Blurring a Cross-Origin Image Client Side Using the HTML5 Canvas

While working as a frontend developer means I get to experiment with new javascript frameworks on at least a yearly basis, it's far less frequent that I get to interact with new, native browser technologies, let alone ones that have wide enough support to be used in production. Thankfully Proton Radio, one of my [amazing] long-running clients has provided the exciting opportunity to experiment with the Web Audio API (more on that later) as well as the HTML5 Canvas in order to implement a variety of features that wouldn't otherwise have been possible.

The Project and the Problem

Proton Radio is an online radio station that continuously streams ~1-2 hour electronic DJ mixes 24 hours a day, and features a custom banner image for most of those mixes (for those that don't have their own banner image we have a variety of fallback images so they still look spiffy). As part of our new Beta redesign we really wanted to minimize the noise by putting these unique banners front and center, and started looking at ways we could make the landing page feel unique to the current DJ mix by applying a matching unique full-page background.

Unfortunately, to make things more complicated all the original image assets were created at 578x291 pixel resolution (some dating from the early-2000's), and stretching them out even on small screens causes them to quickly become extremely pixelated. While we hope to create new banners at higher resolutions in the future, redoing the entire back catalog of thousands of images just wouldn't be practical.

Considering the Options

We initially discussed extracting the color profile from the image to a CSS gradient to the background, but this seemed like a halfway solution as it wouldn't have nearly as much depth as the images itself. Instead, we decided to explore the possibility of scaling up and applying a gaussian blur to the image itself as it would both mask any evidence of pixelation and would give the page a great sense of depth between the foreground and background.

Process Images Server-Side

The most obvious solution was to process all the banner images server-side to have a new 1920x1080, pre-blurred copy that we could then apply as a fullscreen CSS background-image with background-size: cover; to keep it in proportion. From a technical standpoint this would actually be easy to implement as open source image processing libraries are plentiful and simple to use, but on the downside our users would suddenly need to download an additional 150-200kb+ image that exists only to add some blurred, superficial ambiance — something we wanted to avoid if at all possible in our mission to keep page load time down.

CSS3 Filters

The second suggested solution was to use the CSS3 Blur filter, but as I suspected support is still pretty spotty, and the performance is abysmal enough to make the fans in my 2015 Macbook Pro sound like a jet engine about to take off. There's also the annoying issue that the blur filter leaves a lighter ring around the image edge due to it assuming the off-canvas area is white/transparent, so you have to do temperamental negative margin hacks inside of a container element with overflow: hidden; set to get it looking right.

I look forward to the day CSS3 filters are production ready, but I'm guessing we've still got at least a few years to go.

Image Processing With Javascript

This left us with our third option of processing the images client-side with javascript — an option I honestly wasn't all too familiar with despite working in javascript on a daily basis, and assumed wouldn't be reasonable from a performance standpoint on older devices.

Looking into it I quickly figured out that we'd likely need to use a <canvas> element, which based on a few different blog posts should allow me to apply almost any image filter imaginable. Unfortunately, up until this point I never completely grasped the purpose of the <canvas> element, so I decided to do some research and try to add some clarity, starting with the Mozilla Developer Network (MDN) Canvas tutorial.

<canvas> is an HTML element which can be used to draw graphics using scripting (usually JavaScript). This can, for instance, be used to draw graphs, make photo composition or simple (and not so simple) animations...

This quick description along with the rest of the article helped, but I still had only a vague understanding and too many unanswered questions. Why do we need an element to draw to when we have the DOM? Why can't we just use SVG? What's better about the <canvas> compared to anything else? It wasn't until I actually started playing around with the canvas in javascript and used some basic drawing methods that it really clicked.

Looking back, I think the <canvas> can be better introduced as “a rectangular DOM element that contains a 2-dimensional grid of pixels, onto which you can draw arbitrary lines, shapes, or graphics using x,y or vector coordinates relative to the <canvas> element itself, without the performance overhead normally involved with manipulating DOM elements.”

Or more simply, it's kinda like MSPaint for the browser, except you draw with code instead of a paintbrush tool. And it's fast...real fast.

Getting Knee Deep in Canvas Land

With all that in mind I quickly came across a few image blurring libraries, the most popular of which is an old-school standalone script called Stackblur.js. At first glance I was a bit wary as its author's website hasn't been updated in years and it lacks an official Github page, but it thankfully turns out there's several popular forks of it on Github, including a NPM module called stackblur-canvas that adds a few extra features. As a plus, this fork also has just short of 500 stars and the occasional Github issue filed, so I can be a little more confident there's some community support to push patches should it needed to be updated in the future.

While setting it up and blurring an image based on the README was surprisingly quick, there were a few gotchas we figured out along the way.

Canvas Elements Are Dumb by Design

Canvas elements are by design unaware of anything you draw in them other than the color value of each pixel. This can be frustrating if you need to get intelligent information out of a canvas element, but its what helps make them so performant compared to manipulating full DOM elements. Rather than keeping track of the position of a bunch of vector elements, its just a X/Y grid with a native API to help you draw to it. The downside of this is it also means that the canvas element is unaware of your original image's aspect ratio, so you have to do those calculations prior to drawing into it and recalculate anytime there's a window resize event.

Canvas Elements Require CORS Headers

Canvas elements require proper CORS headers on images to be set prior to drawing, otherwise the canvas becomes “tainted” and you can no longer read information out of it due to the potential security implications. While we aren't reading out any information for our application in an obvious way, behind the scenes it turns out Stackblur actually applies the image to the canvas, reads out all the pixel data, calculates the new blurred values, and then applies that over the original image.

The good news is that setting CORS headers for a specific directory is actually really simple in most HTTP servers, and for us amounted to adding the three lines of code below to our Apache httpd.conf file (in many cases this will/should actually be a separate .conf file in the /etc/apache2/sites-available/ folder specific to the VirtualHost in question). You'll also likely need to set the img.crossorigin attribute to '' or 'anonymous' in order to prevent it from having issues in certain browsers (you can read more about CORS enabled images on MDN's article), though browser support for this attribute seems to be inconsistently implemented at best.

Android 4.4 and Below Don't Support Image CORS Rules

Android 4.4 and below aren't a huge part of our user base, but at ~2.5% its still enough traffic to be concerned when there's a application-breaking bug present. In our case we kept getting a cryptic “Uncaught Error: unable to access image data: Error: unable to access local image data: Error: SECURITY_ERR: DOM Exception 18” reported in Bugsnag^[1] for old Android devices that didn't seem to be accurately working with sourcemaps and had no clear answer we could find on StackOverflow. As luck would have it though, I temporarily disabled the background blur for the better part of an afternoon and the error suddenly disappeared. With an Android emulator I was able to reproduce the error, which revealed a less-cryptic alert message I was then able to trace back to the uncompiled Stackblur.js library. I thought it might have to do with the outdated netscape.security.PrivilegeManager rules (the stackblur-canvas package has since removed these), but manually omitting them didn't seem to make a difference.

While I couldn't find any answers via Google, I finally looked deeper into CORS rules and support to see if I was missing something — turns out that Android 4.3 and below specifically don't support CORS for images in a canvas — exactly what we were trying to do! You'd think this would be a common point of discussion, but with Android <4.3 being on its way out I ended up discovering it in the footnotes of the CanIUse CORS page (only shows up “Show All” is enabled).

If you're serving images from the same domain as your client application this of course won't matter, but in our case changing the infrastructure to serve our flyers off the same domain as our Beta, Staging, and local environments wasn't worth the extra complexity it'd introduce to our already-complex infrastructure setup.

After some deliberation we agreed that not having a blurred background image for older Android users was an acceptable level of “graceful degradation”, so I added a simple method to detect Android browser versions and skip over the backgroundBlur method for those with problems^[1:1]. Despite caniuse.com showing that Android 4.4 should support CORS rules on images, we noticed that about half our Android 4.4 users were experiencing problems so we opted to exclude them also.

Chaining it All Together

After a lot of research, some trial and error, and a bit of luck, we pushed our last update to the background blur several weeks back and haven't seen any related bugs since. One last minute change was to hide the <canvas> element with the CSS opacity property until after it finishes blurring, then animate it in using the CSS transition property — this really tied it together and made the whole experience feel more elegant than if it flickered in and out until it was properly blurred.

The screenshot below is what it looks like right now, but you can check it out live at Proton Beta.

For readability's sake here's a Gist with all the relevant code from my Marionette/Backbone view (excludes the Apache CORS rules). Depending on your choice of framework/tools it'll likely need to be adapted, but it should be a good starting point for most views written in javascript.

Notice something amiss or want to chat? Feel free to get in touch with me at rob@atomidesign.com!