Tag Archives: TopoJSON

Building the HumanGeo Website Map

HumanGeo is a relatively new company – the founding date is a little bit fuzzy, but 4 years old seems to be the prevailing notion.  Our website hadn’t changed much in three plus years, and it was looking a little bit dated, even by three year old website standards, so last year we decided to take the plunge and update it.

Screen Shot 2015-06-22 at 7.08.36 AM
Here’s an Internet Archive snapshot of the old site, complete with shadowy HumanGeo guy (emphasis on the “Human” in HumanGeo, perhaps?)

It’s not that the old site was bad, it just wasn’t exciting, and it didn’t seem to capture all the things that make HumanGeo unique – it just sort of blended in with every other corporate website out there.  In building the new website, we set out to create an experience that is a little less corporate and a little more creative, with a renewed focus on concisely conveying what it is we actually do here.  Distilling all of the things we do across projects into a cohesive, clear message was a challenge on par with building the website itself, but I’ll save that discussion for another time.   Since HumanGeo is a young company that operates primarily in the services space, our corporate identity, while centered on a few core themes, is constantly evolving – very much driven by the needs of our partners and the solutions we collectively build.  One of the core themes that has remained despite the subtle changes in our identity over time is our focus on building geospatial solutions.  It’s the “Geo” in HumanGeo; this is precisely why we chose to display a map as the hero image on our main page (sorry HumanGeo guy).  This article is all about building that map, which is less a static image and more a dynamic intro into HumanGeo’s world.  My goal with this article is to highlight the complexity that went into building the map and some of the ways in which the Leaflet DVF and a few other libraries (Stamen, TopoJSON, Parallax JS) simplified development or added something special.

The HumanGeo Website Map

When you arrive at our site, you’re greeted with a large Leaflet JS-based map with a Stamen watercolor background and some cool parallax effects (at least I think it’s cool).  It’s more artsy than realistic (using the Stamen watercolor map tiles pretty much forced our hand stylistically), an attempt to emphasize our creative and imaginative side while still highlighting the fact that geospatial is a big part of many of the solutions that we build.  The map is centered on DC with HumanGeo’s office in Arlington in the lower-left and some pie charts and colored markers in the DC area. The idea here is that you’re looking at our world from above, and it’s sort of a snapshot in time.  At any given point in time there are lots of data points being generated from a variety of sources (e.g. people using social media, sensors, etc.), and HumanGeo is focused on helping our partners capture, analyze and make sense of those data points.  As you move your mouse  over the map or tilt your device, the clouds, satellite, plane, and map all move – simulating the effect of changing perspective that you might see if you were holding the world in your hands.  In general, I wanted this area (the typical hero image/area) to be more than just a large static stock image that you might see on many sites across the web – I wanted it to be dynamic.  If you spend any time browsing our site, you’ll see that this dynamic hero area is a common element across the pages of our site.

Parallax

Parallax scrolling is popular these days, making its way into pretty much every slick, out of the box website template out there.  Usually you see it in the form of a background image that scrolls at a different speed than the main text on a page.  It gives the page dimension, as if the page is composed of different layers in three dimensional space.  In general, I’m a fan of this effect.  However, I decided to go with a different approach for this redesign – mainly because I saw a cool library that applied parallax effects based on mouse or device movement rather than scrolling, and the thought of combining it with a map intrigued me.  The library I’m referring to is Parallax JS, a great library that makes these type of effects easy.  To create a parallax scene, you simply add an unordered list (ul) element – representing a scene – to the body of your page along with a set of child list item (li) elements that represent the different layers in the scene.  Each li element includes a data-depth attribute that ranges from 0.0 to 1.0 and specifies how close to or far away from the viewer that layer will appear (closeness implies that the movement will be more exaggerated).  A value of 0.0 indicates that the layer will be lowest in the list and won’t move, while a value of 1.0 means that the layer will move the most.   In this case, my scene is composed of the map as the first li element with various layers stacked on top of it including the satellite imaging boxes, the satellite, the clouds, and the plane.  The map has a data-depth value of 0.05, which means it will move a little, and each layer over top of the map has an increasing data-depth value.  Here’s what the markup looks like:


<ul id="scene">
<li class="layer maplayer" data-depth="0.05">
 <div id="map"></div>
</li>
<li class="layer imaginglayer" data-depth="0.30">
 <div style="opacity:0.2; top: 190px; left: 1100px; width: 400px; height: 100px;" />
</li>
<li class="layer imaginglayer" data-depth="0.40">
 <div style="opacity:0.3; top: 180px; left: 1150px; width: 300px; height: 75px;" />
</li>
<li class="layer imaginglayer" data-depth="0.50">
 <div style="opacity:0.4; top: 170px; left: 1200px; width: 200px; height: 50px;" />
</li>
<li class="layer imaginglayer" data-depth="0.60">
 <div style="opacity:0.5; top: 160px; left: 1250px; width: 100px; height: 25px;" />
</li>
<li class="layer cloudlayer" data-depth="0.60"></li>
<li class="layer planelayer" data-depth="0.90"></li>
<li class="layer satellitelayer" data-depth="0.95">
 <img src="images/satellite.png" style="width: 200px; height: 200px; top: 20px; left: 1200px;">
</li>
</ul>

A Moving Plane

On modern browsers, the plane moves via CSS 3 animations, defined using keyframes.  Keyframes establish properties that will be applied to the HTML element being animated at different points in the animation.  In this case, the plane will move in a linear path from 0,  800 to 250, -350 through the duration of the animation.


@keyframes plane1 {

     0% {
          top: 800px;
          left: 0px;
     }

     100% {
          top: -350px;
          left: 250px;
     }

}

@-webkit-keyframes plane1 {

     0% {
          top: 800px;
          left: 0px;
     }

     100% {
          top: -350px;
          left: 250px;
     }

}

@-moz-keyframes plane1 ...

@-ms-keyframes plane1 ...

#plane-1 {
     animation: plane1 30s linear infinite normal 0 running forwards;
     -webkit-animation: plane1 30s linear infinite normal 0 running forwards;
     -moz-animation: plane1 30s linear infinite normal 0 running forwards;
     -ms-animation: plane1 30s linear infinite normal 0 running forwards;
}

On to the Map

The map visualizations use a mix of real and simulated data.  Since we’re located in the DC area, I wanted to illustrate displaying geo statistics in DC, but first, I actually needed to find some statistics related to DC.  I came across DC’s Office of the City Administrator’s Data Catalog site that lists a number of statistical data resources related to DC.  Among those resources are Comma Separated Value (CSV) files of all crime incident locations for a given year, such as this file, which lists crime incidents for 2013.  The CSV file categorizes each incident by a number of attributes and administrative boundary categories, including DC voting precinct.  I decided to use DC voting precincts as the basis for displaying geo statistics, since this would nicely illustrate the use of lower level administrative boundaries, and there are enough DC voting precincts to make the map interesting versus using another administrative boundary level like the eight wards in DC.  I downloaded the shapefile of DC precincts from the DC Atlas website and then used QGIS to convert it to GeoJSON to make it easily parseable in JavaScript.  At this point, I ran into the first challenge.  The resulting GeoJSON file of DC precincts is 2.5 MB, which is obviously pretty big from the perspective of ensuring that the website loads quickly and isn’t a frustrating experience for end users.

To make this file useable, I decided to compress it using the TopoJSON library. TopoJSON is a clever way of compressing GeoJSON, invented by Mike Bostock of D3 and New York Times visualization fame.  TopoJSON compresses GeoJSON by eliminating redundant line segments shared between polygons (e.g. shared borders of voting precinct boundaries). Instead of each polygon in a GeoJSON file sharing redundant line segments, TopoJSON stores the representation of those shared line segments, and then each polygon that contains those segments references the shared segments. It does amazing things when you have GeoJSON of administrative boundaries where those boundaries often include shared borders. After downloading TopoJSON, I ran the script on my DC precincts GeoJSON file:

topojson '(path to GeoJSON file)' -o '(path to output TopoJSON file)' -p

Note that the -p parameter is important, as it tells the TopoJSON script to include GeoJSON feature properties in the output.  If you leave this parameter off, the resulting TopoJSON features won’t include properties that existed in the input GeoJSON.  In my case, I needed the voting_precinct property to exist, since the visualizations I was building relied on attaching statistics to voting precinct polygons, and those polygons are retrieved via the voting_precinct property.  Running the TopoJSON script yielded a 256 KB file, reducing the size of the input file by 90%!

It’s important to note that in rare cases using TopoJSON doesn’t make sense.  You’ll have to weigh the size of your GeoJSON versus including the TopoJSON library (a light 8 KB minified) and the additional processing required by the browser to read GeoJSON out of the input TopoJSON.  So, if you have a small GeoJSON file with a relatively small number of polygons, or the polygon features in that GeoJSON file don’t share borders, it might not make sense to compress it using TopoJSON.  If you’re building choropleth maps using more than a few administrative boundaries, though, it almost always makes sense.

Once you have TopoJSON, you’ll need to decode it into GeoJSON to make it useable.  In this case:


precincts = topojson.feature(precinctsTopo, precinctsTopo.objects.dcprecincts);

precinctsTopo is the JavaScript variable that references the TopoJSON version of the precinct polygons.  precinctsTopo.objects.dcprecincts is a GeoJSON GeometryCollection that contains each precinct polygon.  Calling the topojson.feature method will turn that input TopoJSON into GeoJSON.

Now that I had data and associated polygons, I decided to create a couple of visualizations.  The first visualization illustrates crimes in DC by voting precinct using a single L.DataLayer instance but provides different visuals based on the input precinct.  For some precincts, I decided to display pie charts sized by total crime, and for others I decided to draw a MarkerGroup of stacked CircleMarker instances sized and colored by crime.

Screen Shot 2014-09-18 at 7.20.11 PM

Screen Shot 2014-09-18 at 7.19.44 PM

The reason for doing this is that I wanted to illustrate two different capabilities that we provide our partners, event detection from social media and geo statistics.  Including these in the same DataLayer instance ensures that the statistical data only get parsed once.  For the event detection layer, I also wanted to spice it up a bit by adding an L.Graph instance that illustrates the propagation of an event in social media from its starting point to surrounding areas.  In this case, I generated some fake data that represent the movement of information between two precincts, where each record is an edge.  Here’s what the fake data look like:


var precinctConnections = [
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 127',
 'cnt': '120'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 142',
 'cnt': '5'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 128',
 'cnt': '89'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 2',
 'cnt': '65'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 1',
 'cnt': '220'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 90',
 'cnt': '28'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 130',
 'cnt': '180'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 89',
 'cnt': '150'
 },
 {
 'p1': 'Precinct 129',
 'p2': 'Precinct 131',
 'cnt': '300'
 }
];

Here’s the code for instantiating an L.Graph instance and adding it to the map.


var connectionsLayer = new L.Graph(precinctConnections, {
     recordsField: null,
     locationMode: L.LocationModes.LOOKUP,
     fromField: 'p1',
     toField: 'p2',
     codeField: null,
     locationLookup: precincts,
     locationIndexField: 'voting_precinct',
     locationTextField: 'voting_precinct',
     getEdge: L.Graph.EDGESTYLE.ARC,
     layerOptions: {
          fill: false,
          opacity: 0.8,
          weight: 0.5,
          fillOpacity: 1.0,
          distanceToHeight: new L.LinearFunction([0, 5], [5000, 1000]),
          color: '#777'
     },
     tooltipOptions: {
          iconSize: new L.Point(90,76),
          iconAnchor: new L.Point(-4,76)
     },
     displayOptions: {
          cnt: {
                displayName: 'Count',
                weight: new L.LinearFunction([0, 2], [300, 6]),
                opacity: new L.LinearFunction([0, 0.7], [300, 0.9])
          }
     }
});

map.addLayer(connectionsLayer);

The locations for nodes in the graph are specified via the precincts GeoJSON retrieved from the TopoJSON file I discussed earlier.  fromField and toField tell the L.Graph instance how to draw lines between nodes.

The pie charts include random slice values.  I could have used the actual data and broken down the crime in DC by type, but I wanted to keep the pie charts simple, and three slices seemed like a good number.  There are more than three crime types that were committed in DC in 2013.  As a bit of an aside, here’s a representation of crimes in DC color-coded by type  using L.StackedRegularPolygonMarker instances.

dccrimes

The code for this visualization can be found in the DVF examples here.  It’s more artistic than practical.  One of the nice features of the Leaflet DVF is the ability to easily swap visualizations by changing a few parameters – so in this example, I can easily swap between pie charts, bar charts, and other charts just by changing the class that’s used (e.g. L.PieChartDataLayer, L.BarChartDataLayer, etc.) and modifying some of the options.

After I decided what information I wanted to display using pie charts, I ventured over to the Color Brewer website and settled on some colors – the 3 class Accent color scheme in the qualitative section here – that work well together while fitting in with the pastel color scheme prevalent in the Stamen watercolor background tiles.  The Leaflet DVF has built in support for Color Brewer color schemes (using the L.ColorBrewer object), so if you see a color scheme you like on the Color Brewer website, it’s easy to apply it to the map visualization you’re building using the L.CustomColorFunction class, like so:

var colorFunction = new L.CustomColorFunction(1, 55, L.ColorBrewer.Qualitative.Accent['3'], {
 interpolate: false
});

I also wanted to highlight some of the work we’ve been doing related to mobility analysis, so I repurposed the running data from the Leaflet DVF Run Map example and styled the WeightedPolyline so that it fit in better with our Stamen watercolor map, using a white to purple color scheme.  The running data look like this:


var data={
 gpx: {
 wpt: [
 {
 lat: '38.90006',
 lon: '-77.05691',
 ele: '4.98652',
 name: 'Start'
 },
 {
 lat: '38.90082',
 lon: '-77.0572',
 ele: '4.9469',
 name: 'Run 001'
 },
 {
 lat: '38.90098',
 lon: '-77.05725',
 ele: '5.23951',
 name: 'Run 002'
 },
 ...
 {
 lat: '38.89517',
 lon: '-77.02822',
 ele: '4.74573',
 name: 'Run 109'
 }
 ]
 }
}

Check out the Run Map example to see how this data can be turned into a variable weighted line or the GPX Analyzer example that lets you drag and drop GPS Exchange Format (GPX) files (a common format for capturing GPS waypoint/trip data) onto the map to see variations in the speed of your run/bike/or other trip in space and time.

To make the map a little more interesting, I added some additional layers for our office, employees, and some boats in the water (the employee and boat locations are made up).  Now that I had the map layers I wanted to use, I incorporated them into a tour that allows a user to navigate the map like a PowerPoint presentation.  I built a simple tour class that has methods for navigating forward and backward.  Each tour action consists of an HTML element with some explanatory text and an associated function to execute when that action occurs.  In this case, the function to execute usually involves panning/zooming to a different layer on the map or showing/hiding map layers.  The details of the tour interaction go beyond collecting, transforming and visualizing geospatial data, so feel free to explore the code here and check out the tour for yourself by visiting our site and clicking on the HumanGeo logo.

We’re hiring!  If you’re interested in geospatial, big data, social media analytics, Amazon Web Services (AWS), visualization, and/or the latest UI and server technologies, drop us an e-mail at info@thehumangeo.com.