Learning D3

My Experience Learning D3

Like most people working in web development over the past couple of years, I've seen more and sites using D3, a popular JavaScript library, for interactive graphics. I had some idea of what was possible, but I'd personally never worked on an application where D3 was needed, and when I'd briefly looked at examples the learning curve seemed steep.

In this article, I summarize my experience learning some basic D3. I've included code examples to show you how to build a graph similar to one that I built for our client.

The Need for D3

When I started here at Planet Argon in September, the first project that I was assigned to was a redesign of the public facing site for our friends at AlphaClone. They came to us with a mockup of several interactive charts that would focus on different details of an index and its performance. We knew we'd need a pie chart, line charts, horizontal and vertical bar charts, and that several of these charts would need to be able to adjust to show different time periods based on user selection.

We knew that D3 would be capable of what we wanted to do - a quick look at the examples gives some idea of the possibilities. The question was what would the learning curve be like for a developer like me, who is fairly experienced with JavaScript and plotting in other languages (mostly Python using Matplotlib) but had never used D3 before.

I'll be showing you how to build a horizontal bar graph with a select element to allow the user to choose what time periods to display. For the sake of this exercise, let's assume that we're comparing returns of three different index funds over 1-year, 3-year, and 5-year periods.

Building the Chart

The first thing I realized about D3 is the amount of control that you have. I hadn't worked with SVG elements before, but each element must be individually drawn and located within the canvas. In previous plotting work, I was used to having a built-in bar chart or line chart method. There's nothing like that in D3, but it gives you the ability to bind data (as a JavaScript array) to SVG elements. Once you do this, you can use the data to control just about any aspect of the graphics that you like.

The SVG element

Mike Bostock, the creator of D3, has written several helpful examples and tutorials. I relied heavily on his tutorial "Let's Make a Bar Chart". One thing that I quickly noticed is how little HTML is involved; most of the HTML tags are created by D3. Initially, you only need an empty svg element.

<div id="chart">
  <svg></svg>
</div>

This is the only piece of the graphic built with HTML. D3 generates the rest of the HTML dynamically and attaches it to the DOM for you.

Adding JavaScript

The rest of the chart will be built using D3 in a script tag. In this example, we'll use a hosted version of D3, but in a real use case you'd likely want to download and minify the library, so that it can be compressed and cached. And although D3 replicates some jQuery functionality, we'll use jQuery for event handling.

Using d3.json to load data

Similar to jQuery's $.ajax method, D3 has a handful of convenience methods for making AJAX requests for data. One of these is the d3.json method, which will automatically parse JSON and return a JavaScript object. In this case, we'll build the chart entirely within the callback function that is passed to the d3.json method. This callback function takes two arguments: an error and data. Here's the initial setup of the script:

<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script type="text/javascript">
d3.json("data.json", function(error, data) {

  // Very basic error handling
  if (error) throw error;

  // Logic to build the chart goes here...
</script>

Now we have a script that makes an AJAX request and loads our data. Here is where the fun begins.

Our data

While we're discussing the data, let's take a look at how it's laid out:

When loaded into JavaScript, we'll have an array of three objects, each with the name of the fund and the returns sorted by time period. This structure will be important while manipulating the data to build the chart.

Initial Chart Layout

Before we can begin building the data-related pieces of the chart, we'll need to define a couple variables for the size and layout of the chart.

// Define global layout variables
var margin = {top: 10, right: 40, bottom: 10, left: 50};
var barHeight = 30;
var width = 400;

These variables are all in pixels, and fairly self explanatory:

• margin defines the margin around the chart
• barHeight defines the height of each bar in the chart
• width defines the width of the entire chart

The next step is to use d3.select to create a selection of our chart SVG element. This will allow us to edit properties and add graphics when we're ready to do so.

// Select the chart and set its size
var chart = d3.select("#chart-svg");
chart.attr("height", barHeight * data.length + margin.top + margin.bottom)
  .attr("width", width + margin.left + margin.right);

Adding Scales

Much of D3's functionality allows us to translate values of data into pixel values that allow us to draw graphics in the correct size and shape, without having to compute pixel values manually. The first piece that we'll need for this are X and Y scales:

// Add scales
var x = d3.scale.linear().range([0, width]);
var y = d3.scale.ordinal().rangeRoundBands([0, barHeight * data.length]);

What we've done here is create two scales: a horizontal X scale, and a vertical Y scale. A scale is a function that will translate a value from the domain to a value within the range.

The X scale is linear, with a range from 0 to the width of the chart. We'll define the domain later - we'll see that it can change depending on the data. The Y scale is ordinal, meaning that any type of value can be plotted (in this case we'll be using the fund names). The rangeRoundBands method ensures that the bars will have even height.

Adding Initial Data

Because we want to be able to have the data update when the time period select is changed, we will build an updateTable function that will be bound to the "change" event of the time period select (more on that later).

Because we just want the update function to change the chart, and not redraw it completely, we'll first build the chart using some placeholder data to create empty SVG elements that will be updated as different time periods are selected. For each data point we'll use a <g> tag to group the corresponding <rect> and <text> tags.

To do this, we'll need to bind our data to our selection. I won't try to explain how this works in this post, as there are already several very good explanations (I found Scott Murray's to be useful). If you are new to D3, I recommend taking some time to read through these. The way that D3 allows you to control DOM elements through your data is perhaps the most important thing that it does.

In our case, we first want to create <g> elements for each of our data points:

// Bind data to graphics elements
var bar = chart.selectAll("g")
  .data(data)
  .enter().append("g")
  .attr("transform", function(d, i) { return "translate(" + margin.left + "," + i * barHeight + ")"; })
;

The translate applied here has the purpose of moving the element margin.left pixels in the X direction (in our case, 50 pixels to the right) and i * barHeight pixels in the Y direction, which will place the bars in the correct position. Here, i refers to the index of the data point; the first data point (with index 0) will be translated 0px, the second will be translated barHeight pixels, and the third 2 * barHeight pixels.

Next, we'll add <rect> and <text> elements within each group:

// Append rect elements
bar.append("rect").attr("height", barHeight - 1);

// Append text elements
bar.append("text").attr("y", barHeight / 2).attr("dy", ".35em");

Here, the variable bar is a selection, and the methods append and attr are operations on the selection. In this case, the append operator returns a new selection of the appended elements, and attr acts on this selection.

Let's take a closer look at these operations. In the first line

bar.append("rect").attr("height", barHeight - 1);

we are appending <rect> elements to each <g> in the selection bar. We then use the attr method to set the height of each of these <rect> elements to barHeight - 1 pixels (we subtract one to give one pixel of padding between the bars).

In the next line, we append text elements to each <g> in bar:

bar.append("text").attr("y", barHeight / 2).attr("dy", ".35em");

Here, we start getting into slightly more complicated pixel geometry, to correctly place the text. There are several ways to do this, and there's not necessarily a "best" way. The y, dy, text anchor, and text size all interact to determine the text location. In this case, we set the y attribute (which places the bottom of the text) to barHeight / 2, which means that the bottom of the text will be centered within the bar. We then use the dy attribute to apply a relative shift of .35em, which is just less than half the height of the text (based on the font size). Thus the text will be centered vertically just above the midpoint of the bar.

Adding a Y domain and axis

The last step before we update the chart with real data is to set the Y domain and add a Y axis. First we'll add the Y domain. Remember from earlier that our variable y is a scale with its range as the height, broken into as many bands as we have data points. Now, we'll define a domain for y as a list of the names of each data point. All that this does is ensure that our data points are plotted in the same order that they appear in our data array.

// Add a Y axis
var yDomain = data.map(function(d) { return d.name; });
y.domain(yDomain);

Next, we want to create an axis on the left of our chart. We can use d3.axis to do this, applying the scale y with the domain and range that we've defined. We'll also set the tick size to 0 to hide the axis ticks.

var yAxis = d3.svg.axis()
  .scale(y)
  .orient("left")
  .tickSize(0, 0)
;

Now that the axis has been created, we'll add a <g> element to the chart and apply the axis using the call method:

chart.append("g")
  .attr("class", "y axis")
  .attr("transform", "translate(" + margin.left + ",0)")
  .call(yAxis);

At this point, we've created empty bars and text elements, and added an axis. Now we'll need to use our data to apply the correctly format these elements.

Updating our chart with a select element

Now we'll add the functionality to dynamically update data based on a select element to pick the time period. First, we'll add the select element to the HTML, before the SVG element.

<select id="data-select">
  <option value="1 Year">1 Year</option>
  <option value="3 Years">3 Years</option>
  <option value="5 Years">5 Years</option>
</select>

<div id="chart">
  <svg></svg>
</div>

Note that the values of the select element match the keys of the returns for each fund: this is key to our functionality, and if we wanted to add or remove time periods from our data, we would need to make corresponding changes here. If not all funds had the same time periods, or they changed more frequently, we could consider dynamically adding the options based on the data, but for the sake of this example that is not necessary.

The updateChart function

We'll now create a function that updates the chart with new data. This function will take one argument, chartData, which is the data to be plotted. While our initial data has returns for all three time periods, chartData will be an array of objects, where each object has two attributes: name for the name of the fund, and value for the value of the currently selected time period. So if the selected time period is 1 Year, then the data passed to the function would be:

[
  { name: "Fund 1", value: 18.76 },
  { name: "Fund 2", value: 12.42 },
  { name: "Fund 3", value: 15.34 }
]

So the next step is to define the function:

function updateChart(chartData) {
  // Logic to update chart goes here
}

The first thing that we'll do is redefine the X domain, based on the current data. For each time period, we'll set the domain so that maximum value within that time period will correspond with a bar that spans the width of the chart. Thus the scale is relative to each time period. Depending on the application, this is something that we may want to change - we could use the absolute maximum across all time periods.

// Update X domain based on new values
var xDomain = [0, d3.max(chartData, function(d) { return d.value; })];
x.domain(xDomain);

Giving the bars the correct size

The next step is to update the <rect> elements to have the correct size. We'll use the data method to bind chartData, and then use D3's transition to smoothly update the data. We'll then use the X scale to give each rectangle the width that corresponds with the fund's value:

// Update rectangle width and positioning
chart.selectAll("rect")
  .data(chartData)
  .transition()
  .attr("width", function(d) { return x(d.value) - x(0); })
;

Note the use of x(d.value) - x(0): x (our scale) is a function that takes a value within the domain, and returns a pixel value based on the range that we defined. In our case, because all values are positive and our domain and range both begin at 0, x(0) will be 0, but this isn't always the case.

Setting the text

The last step involved with updating the chart is updating the text that displays the value, and placing it correctly. We want the text to be within the bar, but in the case of a low value where the bar is not large enough to contain the text, we'll place the text to the right of the bar. As with the <rect> elements, we'll use the data and transition methods:

chart.selectAll("text")
  .data(chartData)
  .transition()
  .attr("x", function(d) {
    // Draw text inside the rectangle if rect is more than 50 px wide
    // Otherwise, draw text just right of the rectangle
    if (x(d.value) - x(0) > 50) {
      return x(d.value) - 50;
    } else {
      return x(d.value, 0) + 3;
    }
  })
  .text(function(d) { return d.value.toFixed(2) + "%"; })
;

Here we have a slightly more complex function to set the x attribute: it determines the width of the rectangle, and draws the text in the appropriate location: 50 pixels left of the end of the rectangle if the rectangle more than 50 pixels wide, and 3 pixels right of the rectangle otherwise. We then use the text method to write the text itself, which is the value rounded to 2 decimal points, and given a percent sign.

This is the end of the updateChart function. Here's the complete function:

function updateChart(chartData) {

  // Update X domain based on new values
  var xDomain = [0, d3.max(chartData, function(d) { return d.value; })];
  x.domain(xDomain);

  // Update rectangle width and positioning
  chart.selectAll("rect")
    .data(chartData)
    .transition()
    .attr("width", function(d) { return x(d.value) - x(0); })
  ;

  chart.selectAll("text")
    .data(chartData)
    .transition()
    .attr("x", function(d) {
      // Draw text inside the rectangle if rect is more than 50 px wide
      // Otherwise, draw text just right of the rectangle
      if (x(d.value) - x(0) > 50) {
        return x(d.value) - 50;
      } else {
        return x(d.value, 0) + 3;
      }
    })
    .text(function(d) { return d.value.toFixed(2) + "%"; })
  ;
}

Binding to the change event

Up until now, all the functionality we've been using has been D3. But if you remember, we also included jQuery. While D3 has event handling functionality, this type of functionality is more easily added with jQuery. We'll bind the chart updating to the change event of the select element. The first step will be to build a new chartData array with the values of the selected time period; this is why the keys must match the values of the select. Once this array is constructed, we'll pass it to the updateChart function and D3 will redraw the chart.

$("#data-select").on("change", function() {
  var timePeriod = $(this).val();
  // Create chartData from currently selected time period values
  var chartData = data.map(function(d) {
    return { name: d.name, value: d.returns[timePeriod] };
  });
  updateChart(chartData);
});
$("#data-select").trigger("change");

Now any time that the select is changed, the chart will update. We also trigger the change to draw the chart initially, with the currently selected value.

Explore D3

If you're interested in more D3 examples, both basic and advanced, check out the tutorials on D3's Github.

All of the code for this example is available as a Github gist and can be seen in action as well.

Thanks for exploring D3 with me!