Mastering D3.js (part 1): Using Canvas to Better Scale Charts

A Scatter Plot Example

Written by Maximiliano Duthey, Full Stack Developer @ XOOR

Without a doubt D3.js has become the go-to technology when it comes to data visualization on a web browser. With D3 you can do almost whatever you can imagine in terms of dynamic and interactive infographics for data visualization on the browser using technologies like SVG, HTML5 and CSS. One of the big pros of D3 is it’s flexibility, which usually gets translated into increased complexity when building new things.

At XOOR we’ve been working lately for one of our customers on a project that required a migration of a set of plots that were implemented with a Python library into a more robust, scalable solution. Some of the main characteristics of the legacy code were:

• Plots were rendered using SVG
• Plots were rendering (very) big amounts of data in the range of ~2 to ~25 megabytes of raw JSON data.
• Plots had different features like zoom and panning, box zooming, tooltips to show values, etc.

The main problem was that using SVG, the browser has to handle a huge amount of data and usability reaches the lowest level a human being could accept (yeah, it’s that bad). Imagine your browser handling millions of elements like this:

<circle cx="25" cy="75" r="20" stroke="red" fill="red"/>

Once you add a few features like zoom and panning, your browser will die.

Once you start reading about D3, you’ll find most examples use SVG to draw. But there’s an alternative which is extremely better for large amounts of data. It comes with added complexity, but once you get used to it and learn how to handle things, you’ll see why it’s much better for some use cases.

Canvas to the rescue!

So what’s a canvas? According to MDN:

Added in HTML 5, the HTML canvas element can be used to draw graphics via scripting in JavaScript. For example, it can be used to draw graphs, make photo compositions, create animations, or even do real-time video processing or rendering.

With canvas, instead of managing millions of elements in the browser, we handle just 1 (well, it’s more than 1 actually as you’ll see later we need some SVG to handle plot axes… but it’s nothing compared to millions), and use it as a whiteboard to draw our plots content on it with a simple but complete Javascript API.

This is the main reason why we chose Canvas as our drawing whiteboard instead of SVG. It wasn’t easy, and it took a considerable amount of time to build the base components, but we consider ourselves at a point where we understand what’s going on. So we thought, why not sharing what we learnt with the people out there! :)

The goal is to write a few posts with some very basic examples of how to build cool things with D3 and HTML canvas. The first one here will help you build the basics of a scatter plot drawing thousands of random points into a D3 canvas.

TL;DR

As always, you’re welcome to skip all our writings and head over to the Github repo where this tutorial is hosted:

https://github.com/xoor-io/d3-canvas-example

And here the working example in Github pages:

https://xoor-io.github.io/d3-canvas-example/

The HTML code

Since we want to keep things simple enough, we’ll build the plots using plain HTML, Javascript and CSS. You can go ahead and build the same with your favorite frontend framework, but we won’t cover details about that in this posts.

The following code will be all we need in our index.html file:

There you can see that we are importing a few things like a styles.css file and two scripts: the D3 v5 script and our plot.js script. Finally we define the scatter plot container as the unique content of our .

The Styles

Our stylesheet looks pretty simple:

As you can see, we will build a plot with size 800x600 pixels and we want it to be centered on the screen. We define then two classes for the main canvas where the points will be drawn and an SVG container on which we’ll be drawing the axes.

Writing some Javascript

Now it’s time to write some Javascript to build our plot. The first thing we have to do is create some random data. The code we’re using for that purpose is pretty simple as you can see in the *plot.js *file:

In this piece of code, we define *dataExample, *which is an array that will contain the points to be drawn on our scatter plot. We’re pushing to it 10000 points defined by the x and y coordinates. These numbers will be totally random within the 1..999999 range.

Next step is to define a few constants to control the look and feel of our plot:

• pointColor: as you probably guessed, this will be the color of the points in our scatter plot.
• margin: an object that defines the margins for the SVG that will contain the axes.
• outterWidth and outterHeight: these define the size of the plot. Usually this will match the same size defined in the main stylesheet for the plot container.
• width and height: these define the size of the “drawable” area inside the plot canvas.

Writing some D3 code

Now it’s time to write D3 related code. Let’s see how that looks like and we’ll explain details afterwards:

1. First thing we do is use the select function (more details here) to get a reference to our scatter plot container. We do this by passing the div class name very similar to how you’d do it with JQuery.
2. Then we create a variable called svgChart. This variable the SVG part of our plot which will mainly contain the plot vertical and horizontal axes. Inside this SVG we append a group (g) element which will contain the axes and we apply a transformation to it so that it has the correct margins.
3. Next is the canvasChart variable that will hold the plot drawable area where all the points will appear. To apply margins in this case we can use the known CSS properties (something we can’t do with SVG and that’s why we used translate in step 2).
4. Finally we assign the 2D context of the canvas to a variable so that we have it there for quick access when drawing in the canvas.

Defining the Scales and Axes

Now we need to take care of the D3 Scales. As the main Github repo for d3-scale explains:

Scales are a convenient abstraction for a fundamental task in visualization: mapping a dimension of abstract data to a visual representation.

D3 scales are the elements that will allow us to define the mapping of those numbers we generated randomly before to a position in pixels for dots in our scatterplot. Let’s see how can we do that:

So let’s take X as an example:

1. First we call the scaleLinear() function to create a (as the name suggests) a linear scale which are the best for continuous quantitative data.
2. Then we define the domain for our scale. The domain tells the scale what are the expected values that will appear on the plot (this is the numbers we generated randomly).
3. Next we tell the scale into which range all those values will be drawn. The range defines the real area on the canvas into which the values within the domain will be drawn. So basically we want our range to go from 0 to the width of our canvas.
4. Finally we call the nice() function so that our scale always starts and ends in round values.

I know it might be confusing, so let’s summarize points 2 and 3:

Domain -> means real data in whatever unit you want to represent it (meters, liters, units, etc).

Range -> means pixels, this is the range of pixels into which we’ll be drawing the points from our Domain.

Notice how for the Y scale we inverted the range to go from height to zero, instead of from zero to height. This is beacuse the canvas origin is on the top left corner and not on the bottom left as the scatter plot, so we need to invert the way data values increase on the Y axes.

In the last two lines we define the axis variables using the previously defined scales. We create the X Axis as a bottom axis, and Y as a left axis.

Appending the Axes

To append the axes to the SVG area, we use the following code:

If you remember, svgChart is a reference to a group within the SVG chart area that was already translated with respect to the plot margins.

The first thing to do is create a new group within the SVG to contain the X axis. Remember that the origin is always on the top left, so we need to translate the X axis so that it’s “pushed” down to the bottom of the plot. Next we do something very similar with the Y axis. In this case we don’t need to translate as it will be positioned on the left side directly.

Finally we add 2 SVG text elements so that we can identify the axes on the plot easily.

Drawing the points

We’re now ready to draw the points on the canvas area. This is the code that does the magic:

We’ve created a function called *drawPoint *that will take care of drawing a single point in the canvas context. The important thing here is to remember that we need to translate the numbers in our domain to pixels on our plot range. This is done using the scale references x and y. Basically by doing:

const px = x(point[0]);

We translate our domain value point[0] into a pixel coordinate within our canvas range.

Once we have the real pixel coordinates of the point, we can use the arc function to draw a circle on the canvas. We use the px and py translated coordinates, as well as the radius, start angle and end angle. Finally we call the fill() function to fill the shape with our point color.

The Result

All set! You can now open the index.html file in your favorite browser (hope it’s not IE) and… voilà!

The final result

Nice! Not very interactive yet though… but hope you learned the basics of how to create charts with D3 using Canvas instead of SVG for high performance on very large datasets.

We’ll be extending this example soon with features to make it more interactive such as zooming, panning, tooltips, etc.

Thanks for reading and we’d love to see your comments and suggestions!

Don’t miss our posts, follow us now on Twitter!