Making maps with D3

Making maps with D3

I used D3 to build a data driven map. The goal was to build a map using D3 using data from a service.

The service provides a JSON file, which consists of place names and a count of documents. The place names are country names or US states, as in the following sample:

[{"id":121,"value":"iran","count":2508},{"id":88,"value":"washington","count":1778}]

Overview

I started with Mike Bostock’s Let’s Make a Map since it was most helpful in getting me to a US map.

The general steps are as follows:

  1. Get shape files
  2. Filter out what you need
  3. Merge and convert to TopoJSON
  4. Build D3 Javascript to join data to the TopoJSON and display the map

1.  Find Shape Files

After much experimentation I found the best shape files to use were from Natural Earth Data. They have three sizes – large 1:10m, medium 1:50m, and small 1:110m.

I found that the large size produced a JSON file that was around 2.4 megabytes, much too large for use in a web browser. The lines drawn for the large map were very smooth. The small shape would produce a JSON file that was 96k, but it was missing a good number of small countries and used more jagged lines. The medium size came out to 618k and contained all of the countries I needed.

2.  Filter Shape Files

For this project, I used Admin 0 Countries and Admin 1 States & Provinces without large lakes. To begin, we need to extract just the US states from the states & provinces shape file, since we only want the US states.

To do this, we use some SQL. First, find the column name that indicates what data is from the USA using ogrinfo.

ogrinfo -sql 'select * from ne_50m_admin_1_states_provinces_lakes' ne_50m_admin_1_states_provinces_lakes.shp -fid 0

This will print out the data in the first row of the shape file, which should be all the data for the first state in the file. Find the column name that indicates the country name. In this case it is _sr_adm0a3. To see if it works with the USA use this:

ogrinfo -sql "select * from ne_50m_admin_1_states_provinces_lakes where sr_adm0_a3 = 'USA'" ne_50m_admin_1_states_provinces_lakes.shp -fid 0

So now we want to convert it to a GeoJSON file using ogr2ogr.

ogr2ogr -f GeoJSON -where "sr_adm0_a3 = 'USA'" states.json ne_50m_admin_1_states_provinces_lakes.shp

Now let’s move on to countries. The first time I tried this, I got all the way to trying to produce the map in the browser and found that it would not add the coloring to represent the data into the countries.

It turns out that this shape file has the country names defined with a column name that is uppercase NAME. The states file had it defined as lowercase name. This is the key to matching up the data in the JSON file. I could tell by running ogrinfo that the column names were different.

ogr2ogr countries.shp ne_50m_admin_0_countries.shp -sql "select NAME as name, POSTAL, ISO_A2, ISO_A3, scalerank, LABELRANK from ne_50m_admin_0_countries"

To change the name of the column use SQL as (shown in the command above). The shape file also contains lots of data I did not need like population and GDP. I eliminated it by only selecting the columns that I wanted to use. This will produce an interim shape file called countries.shp.

Next, convert the countries shape files into a GeoJSON file using ogr2ogr.

ogr2ogr -f GeoJSON countries.json countries.shp

3. Convert to TopoJSON

The goal is to get a topoJSON file since it stores the data most efficiently. This next command will convert the two GeoJSON files (states and countries) into one TopoJSON file by merging the data together. (Remember that we have named the countries of the world as countries and the US states are called states.)

topojson --id-property name --allow-empty -o world.json countries.json states.json

The –id-property setting will make the name field the id. This is used to join the data from the document count JSON file. The allow-empty setting forces it to save the polygons for all the countries even if they are very small. Without that setting, I found that TopoJSON would remove some of the small geographical entities that are countries or territories like Aruba. See the TopoJSON documentation for more info.

4. Build D3 Javascript

The next step is to build the html page and javascript that will draw the map using our data. If you prefer to skip to the completed code use this link.

First of course, we must have the TopoJSON and D3 javascripts loaded.

Next, set up the map projection and size of the map.

<script> var width = 960, height = 960; var projection = d3.geo.mercator().scale(200); var path = d3.geo.path().projection(projection); var svg = d3.select("body").append("svg").attr("width", width).attr("height", height); var g = svg.append("g");

The queue command will set up the loading of the two JSON files that will be merged.
queue()
.defer(d3.json, "world.json")
.defer(d3.json, "toplocations.json")
.await(ready);

Now, here’s the main function that is used to draw everything.
function ready(error, world, locations) {
console.log(world)

In the style section, we need to add the following styles in order to draw the lines of the map.

.subunit-boundary {
fill: none;
stroke: #777;
stroke-linejoin: round;
}

The console command will output the contents to the browser console which you can view using Firebug or the JavaScript console in many browsers. Inside of this function, put the following code to draw the boundaries of the continents. Note that I am referencing world.objects.countries; this is where each country is stored in the world.json file.

Listing of the countries in the world.json

g.append("path")
.datum(topojson.mesh(world, world.objects.countries, function(a, b) { return a == b }))
.attr("d", path)
`.attr(“class”, “subunit-boundary”);“

 

Ocean borders sans countries

This next one draws the lines between each country.

g.append("path")
.datum(topojson.mesh(world, world.objects.countries, function(a, b) { return a !== b }))
.attr("d", path)
.attr("class", "subunit-boundary");

Country Borders

Now add the US states and the little bit around the great lakes.

g.append("path")
.datum(topojson.mesh(world, world.objects.states, function(a, b) { return a !== b }))
.attr("d", path)
.attr("class", "subunit-boundary");
g.append("path")
.datum(topojson.mesh(world, world.objects.states, function(a, b) { return a == b }))
.attr("d", path)
.attr("class", "subunit-boundary");
};

US States

The next step involves converting the count on each country into a color, finding the matching state or country, and filling in the color. This is done using D3 and CSS.

Add the following code to the style section, which will control what colors go in each range of values indicated for each country. Again, thanks to Mike Bostock for this piece of code. The .subunit class is used to fill in the regions that do not have a count value in the toplocations.json.

.subunit { fill: #aaa; }
.q0-9 { fill:rgb(247,251,255); }
.q1-9 { fill:rgb(222,235,247); }
.q2-9 { fill:rgb(198,219,239); }
.q3-9 { fill:rgb(158,202,225); }
.q4-9 { fill:rgb(107,174,214); }
.q5-9 { fill:rgb(66,146,198); }
.q6-9 { fill:rgb(33,113,181); }
.q7-9 { fill:rgb(8,81,156); }
.q8-9 { fill:rgb(8,48,107); }

Next is the javascript to run through the toplocations.json file and make a map of every country and the count. This is a loop that will iterate through each country and put its name and count in a map. This is used later to match up the country name (id) in the world.json file and find the count.

locations.forEach(function(data) {
countByName.set(data.value, data.count);
})

We also need this quantization code outside of the main function block. Open this window to see where it is placed in the code.

var quantize = d3.scale.quantize()
.domain([0, 2000])
.range(d3.range(9).map(function(i) { return "q" + i + "-9"; }));

This will take the count from each country and turn it into a range from 1 – 9. It will only handle values up to 2000 because of the domain command.

Next, we need similar code to the one we used to draw the borders. One is for the countries, and one is for the states because they ended up in two different arrays in the world.json.

g.selectAll(".countries")
.data(topojson.feature(world, world.objects.countries).features)
.enter().append("path")
.attr("class", function(d) { return "subunit " + quantize(countByName.get(d.id.toLowerCase())); })
.attr("d", path);
g.selectAll(".states")
.data(topojson.feature(world, world.objects.states).features)
.enter().append("path")
.attr("class", function(d) { return "subunit " + quantize(countByName.get(d.id.toLowerCase())); })
.attr("d", path);

Note the countByName function which finds the id from the map JSON. This is the country or state name. It must be changed to lowercase so it will match up with the data in our toplocations.json file. It will return the count for that country and then quantize will convert it to a CSS class that corresponds to a color. This class is added to the JSON so that when the browser draws, it will be filled with the correct color.

Now for the cream on top. It is always nice to allow for zooming and movement of the map. The following code will allow your map users to control their point of view.

var zoom = d3.behavior.zoom()
.on("zoom",function() {
g.attr("transform","translate("+
d3.event.translate.join(",")+")scale("+d3.event.scale+")");
});
svg.call(zoom)

Your Turn

Hopefully, this example will help you in building your own data driven map. All the code and a working sample can be found at http://bl.ocks.org/bradllj/8326068.

References

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *