OpenLayers 3 and Vector Data

As part of a project to move most of my OpenLayers 2-powered websites to OpenLayers 3, I have recently converted two more – DataShine: Travel to Work Flows and the North/South Interactive Map. Unlike the main DataShine: Census website, both of these newer conversions include vector geospatial data, so there was additional learning involved during the migration process, mainly relating to vector styling.

northsouth2North/South Interactive Map

For the North/South Interactive Map, I made use of the loading in of remote GeoJSON files.

Vector Layers

Here’s a vector layer:

layerPoints = new ol.layer.Vector({
    source: pointSource,
    style: function(feature, res) { return pointStyle(feature, res); }

The pointSource is a ol.source.GeoJSON, which requires the projection of the files to be defined, as well as that to be displayed, when defining the source for the Vector layer:
pointSource = new ol.source.GeoJSON({
    url: '...',
    defaultProjection: 'EPSG:4326',
    projection: 'EPSG:3857',

    attributions: [ new ol.Attribution({ 'html': "..." }) ]

If you wish to do further operations on your data once it is loaded in, you need to add a listener to a remotely loaded (e.g. GeoJSON file) source included within a Vector layer:

pointSource.once('change', function()
    if (pointSource.getState() == 'ready')
    { var features = pointSource.getFeatures(); ... }

Here’s a typical style function. I’m using a property “highlight” on my feature to style such features differently:

function pointStyle(feature, resolution)
    return [
            image: new{
                radius: (feature.highlight ? 7 : feature.radius ),
                fill: new{ color: feature.fillColor }),
                stroke: new{ width: feature.strokeWidth, color: '#fff' })
            text: new{
                text: (feature.highlight ? feature.label : ""),
                font: '9px Ubuntu, Gill Sans, Helvetica, Arial, sans-serif',
                fill: new{ color: '#fff' })


To detect clicks, I used an ol.interaction.Select – N.B. if you don’t specify which layers it applies to, it tries to apply them to all Vector layers!

var selectClick = new ol.interaction.Select({
    style: function(feature, res) { return pointStyle(feature, res); },
    layers: [layerPoints]

selectClick.getFeatures().on('change:length', function(e)
{ ... }


In my function here I remove the flag from any already highlighted features and call features[i].changed(); to get the non-highlighed style. You don’t need to call this on what you’ve actually clicked on, as this is done implicitly. here’s likely better ways of showing selected/highlighted features, using ol.FeatureOverlay, but i couldn’t get this to work.



There’s quite a nice new utility function which means it was little effort to get an “old style” location indicator in, at the bottom of the North/South interactive:
new ol.control.MousePosition({ projection: "EPSG:4326", coordinateFormat: ol.coordinate.toStringHDMS, className: 'olControlMousePosition' })


DataShine: Travel to Work Flows

This loads vector data in as generic JSON through a regular (non-OL) AJAX call rather than GeoJSON so the processing is a bit more manual. This time, my source for the Vector layer is a simple ol.source.Vector which can be emptied with source.clear(); and reused.

I’m creating lines directly from the JSON, converting from OSGB grid and specifying colour (for the style) as I go – note my use of rgba format, allowing me to specify a partial transparency (of 60%) for the lines:

var startLL = ol.proj.transform([data[start][2], data[start][3]], "EPSG:27700", "EPSG:3857");
var endLL = ol.proj.transform([data[end][2], data[end][3]], "EPSG:27700", "EPSG:3857");
var journeyLine = new ol.geom.LineString([startLL, endLL]);
var lineItem = new ol.Feature({ geometry: journeyLine });
lineItem.strokeColor = 'rgba(255, 0, 0, 0.4)'; lineSource.addFeature(lineItem);

As previously blogged, I’m also using hand-crafted permalinks in both websites, and drag-and-drop KML display and UTF grid mouseovers in the latter, and both have also had their stylesheets tweaked to allow for easy printing – again made possible with OL3.

I’m about ready now to tackle my most complicated OpenLayers project by far, the Bike Share Map.

Visit the new Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

OpenLayers 3 and DataShine


OpenLayers is a powerful web mapping API that many of my websites use to display full-page “slippy” maps. DataShine: Census has been upgraded to use OpenLayers 3. Previously it was powered by OpenLayers 2, so it doesn’t sound like a major change, but OL3 is a major rewrite and as such it was quite an effort to migrate to it. I’ve run into issues with OL3 before, many of which have since been resolved by the library authors or myself. I was a bit grumbly in that earlier blogpost for which I apologise! Now that I have fought through, the clouds have lifted.

Here are some notes on the upgrade including details on a couple of major new features afforded by the update.

New Features

Drag-and-drop shapes

One of the nicest new features of OL3 is drag-and-dropping of KMLs, GeoJSONs and other geo-data files onto the map (simple example). This adds the features pans and zooms the map to the appropriate area. This is likely most useful for showing political/administrative boundaries, allowing for easier visual comparisons. For example, download and drag this file onto DataShine to see the GLA boundary appear. New buttons at the bottom allow for removal or opacity variation of the overlay files. If the added features include a “name” tag this appears on the key on the left, as you “mouse over” them. I modified the simple example to keep track of files added in this way, in an ol.layer.Group, initially empty when added to the map during initialisation.

Nice printing

Another key feature of OL3 that I was keen to make use of is much better looking printing of the map. With the updated library, this required only a few tweaks to CSS. Choosing the “background colours” option when printing is recommended. Printing also hides a couple of the panels you see on the website.

Nice zooming

OL3 also has much smoother zooming, and nicer looking controls. Try moving the slider on the bottom right up and down, to see the smooth zooming effect. The scale control also changes smoothly. Finally, data attributes and credits are now contained in an expandable control on the bottom left.

A bonus update, unrelated to OL3, is that I’ve recreated the placename labels with the same font as the DataShine UI, Cabin Condensed. The previous font I was using was a bit ugly.

Major reworkings to move from OL2 to OL3

UTF Grids

With OpenLayers 3.1, that was released in December 2014, a major missing feature was added back in – support for UTF Grid tiles of metadata. I use this to display the census information about the current area as you “mouse over” it. The new implementation wasn’t quite the same as the old though and I’ve had to do a few tricks to get it working. First of all, the ol.source.TileUTFGrid that your UTF ol.layer.Tile uses expects a TileJSON file. This was a new format that I hadn’t come across before. It also, as far as I can tell, insists on requesting the file with a JSONP callback. The TileJSON file then contains another URL to the UTF Grid file, which OL3 also calls requiring a JSONP callback. I implemented both of these with PHP files that return the appropriate data (with appropriate filetype and compression headers), programmatically building “files” based on various parameters I’m sending though. The display procedure is also a little different, with a new ol.source.TileUTFGrid.forDataAtCoordinateAndResolution function needing to be utilised.

In my map initialisation function:

layerUTFData = new ol.layer.Tile({});

var handleUTFData = function(coordinate)
  var viewResolution = olMap.getView().getResolution();
  layerUTFData.getSource().forDataAtCoordinateAndResolution(coordinate, viewResolution, showUTFData);

$(olMap.getViewport()).on('mousemove', function(evt) {
  var coordinate = olMap.getEventCoordinate(evt.originalEvent);

In my layer change function:

layerUTFData.setSource(new ol.source.TileUTFGrid({
  url: "" + jsonName

(where jsonName is how I’ve encoded the current census data being shown.)


var callback = function(data) { [show the data in the UI] }

In utf_tilejsonwrapper.php:

header('Content-Type: application/json');
$callback = $_GET['callback'];
$json_name = $_GET['json_name'];
echo $callback . "(";
echo "
{ 'grids' : ['{x}&y={y}&z={z}&json_name=$json_name'],
'tilejson' : '2.1.0', 'scheme' : 'xyz', 'tiles' : [''], 'version' : '1.0.0' }";
echo ')';

(tilejson and tiles are the two mandatory parts of a TileJSON file.)

In utf_tilefilewrapper.php:

header('Content-Type: application/json');
$callback = $_GET['callback'];
$z = $_GET['z'];
$y = $_GET['y'];
$x = $_GET['x'];
$json_name = $_GET['json_name'];
echo $callback . "(";
echo file_get_contents("http://[URL to my UTF files or creator service]/$json_name/$z/$x/$y.json");
echo ')';


The other change that required careful coding to recreate the functionality of OL2, was permalinks. The OL3 developers have stated that they consider permalinks to be the responsibility of the the application (e.g. DataShine) rather than the mapping API, and, to a large extent, I agree. However OL2 created permalinks in a particular way and it would be useful to include OL3 ones in the same format, so that external custom links to DataShine continue to work correctly. To do this, I had to mimic the old “layers”, “zoom”, “lat” and “lon” parameters that OL2’s permalink updated, and again work in my custom “table”, “col” and “ramp” ones.

Various listeners for events need to be added, and functions appended, for when the URL needs to be updated. Note that the “zoom ended” event has changed its name/location – unlike moveend (end of a pan) which sits on your, the old “zoomend” is now called change:resolution and sets on olMap.getView(). Incidentally, the appropriate mouseover event is in an OL3-created HTML element now – olMap.getViewport() – and is mousemove.

Using the permalink parameters (args):

if (args['layers']) {
  var layers = args['layers'];
  if (layers.substring(1, 2) == "F") {
[& similarly for the other args]

On map initialisation:

args = []; //Created this global variable elsewhere.
var hash = window.location.hash;
if (hash.length > 0) {
  var elements = hash.split('&');
  elements[0] = elements[0].substring(1); /* Remove the # */
  for(var i = 0; i < elements.length; i++) {
    var pair = elements[i].split('=');
    args[pair[0]] = pair[1];

Whenever something happens that means the URL needs an update, call a function that includes this:

var layerString = "B"; //My old "base layer"
layerBuildMask.getVisible() ? layerString += "T" : layerString += "F";
layerString += "T"; //The UTF data layer.
var centre = ol.proj.transform(olMap.getView().getCenter(), "EPSG:3857", "EPSG:4326");
window.location.hash = "table=" + tableval + "&col=" + colval + "&ramp=" + colourRamp + "&layers=" + layerString + "&zoom=" + olMap.getView().getZoom() + "&lon=" + centre[0].toFixed(4) + "&lat=" + centre[1].toFixed(4);

Issues Remaining

There remains a big performance drop-off in panning when using DataShine on mobile phones and other small-screen devices. I have put in a workaround "viewport" meta-tag in the HTML which halves the UI size, and this makes panning work on an iPhone 4/4S, viewed horizontally, but as soon as the display is a bit bigger (e.g. iPhone 5 viewed horizontally) performance drops off a cliff. It's not a gradual thing, but a sudden decrease in update-speed as you pan around, from a few per second, to one every few seconds.

Additional Notes

Openlayers 3 is compatible with Proj4js version 2 only. Using this newer version requires a slightly different syntax when adding special projections. I use Proj4js to handle the Ordnance Survey GB projection (aka ESPG:27700), which is used for the postcode search, as I use a file derived from the Ordnance Survey's Code-Point Open product.

I had no problems with my existing JQuery/JQueryUI-based code, which powers much of the non-map part of the website, when doing the upgrade.

Remember to link in the new ol.css stylesheet, or controls will not display correctly. This was not needed for OL2.

OL3 is getting there. The biggest issue remains the sparsity of documentation available online - so I hope the above notes are helpful in the interim.


Above: GeoJSON-format datafiles for tube lines and stations (both in blue), added onto a DataShine map of commuters (% by tube) in south London.

Visit the new Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

North/South – The Interactive Version.


As a weekend project, I’ve made an interactive version of my London North/South artwork.

As well as the blue and red house silhouettes, assembled in QGIS, I’ve added in GeoJSON files of the River Thames (from Ordnance Survey Vector Map District, like the buildings) and of tube/DLR/Overground stations – the location/name/network data is from this GitHub file and I’ve applied a custom styling in OpenLayers 2, with station name styling inspired by the NYC Subway signs. The positional information comes from an OpenLayers control – I’m using a utility function to modify the output to use degrees, minutes and seconds. Finally, the naming popup is a set of UTFGrid JSON files (with 2-pixel resolution) based on OpenStreetMap data for polygons. Where the polygon has a building, leisure or waterway tag, I’m extracting a name, if available, and showing it. The coverage here is therefore only as good as building naming is in OpenStreetMap. I could potentially add in street names in the future.

Try it out here.

Primary Roads


Britain’s “top” primary roads – the A1, A2, A3… to A9 – are arranged in a particular pattern, with the A1-A6 radiating out clockwise from London and the A7 to A9 similarly radiating around Edinburgh.

I used Gemma, an old UCL CASA project that Steve and I worked on back in 2011, to draw, from OpenStreetMap, the routes of the A1-A6 as they leave London. The A5 has a gap between Edgware and Harpenden, and the A6 only starts at Luton – both of these changes likely due to the building of the M1 motorway which effectively replaced those sections. Co-numbered roads are not included in the map due to a conflict with the way OpenStreetMap and Gemma separate information. Key for the maps: Red = A1, Orange = A2, Green = A3, Blue = A4, Purple = A5, Black = A6.

Also of interest is that the only two roads that “touch” in London are the A2 and A3, at Borough. The other roads may at one time have converged at junctions, but their starts have been shortened slightly over the years. The big junction at Bank certainly looks like a place where the A1, A3 and A4 could have started from. (Outside of London, the A7 touches the A1 at its northern end and the A6 at its southern end.) Diamond Geezer walked the first mile of the A1-A5 a few years ago.

Gemma still partially works, despite not having seen much love for the last few years and having never made it out of beta (it was a short project). It is recommended you use the OpenStreetMap (or Marker) layers only, to avoid bugs, and watch out if removing layers. You can see the live A1-A6 map here or have a go at building your own.


Key for the maps: Red = A1, Orange = A2, Green = A3, Blue = A4, Purple = A5, Black = A6.

I’ve blogged about Gemma before (more).

The coloured road lines are Copyright OpenStreetMap contributors and the greyscale background map is Copyright Google.

All the Tweets

Cross-posted from Mapping London, edited slightly.

This is a map of geolocated Tweets for the whole world – I’ve zoomed into London here. The map was created by Eric Fischer of Mapbox, who collected the tweets over several years. The place where each tweet is posted from is shown by a green dot. There are millions and millions of tweets on the global map – in fact, over 6.3 billion. The map is zoomable and the volume of tweets means that popular locations stand out even at a high zoom level. The dots are in fact vectors, so retain their clarity when you zoom right in. The map is interactive – pan around to explore it.

If you think this looks familiar, you’d be right. Mapping London has featured this kind of social media ‘dot-density mapping’ a few times before, including with Foursquare and Flickr (also Eric’s work), as well as colouring by language. The key difference with this latest map is the sheer volume of data. By collecting data on geolocated tweets over the course of several years, globally, Eric has assembled the most comprehensive map yet. He has also taken time to ensure the map looks good at multiple zoom levels, by varying the dot size and dot density. He’s also eliminated multiple tweets that happen at the exact same location, and reduced some of the artefacts and data quality issues (e.g. straight lines of constant latitude or longitude) to produce perhaps the cleanest Twitter dot-density map yet. Zooming out makes the map appear somewhat similar to the classic night-time satellite photos of the world, with the cities glowing brightly – here, London, Paris and Madrid are prominent:


However it should still be borne in mind that while maps of tweets bear some relationship to a regular population density map at small scales, at large scales they will show a bias towards places where Twitter users (who may be more likely to be affluent and younger than the general population) live, work and socialise. The popularity of the social network also varies considerably on a country-by-country basis. Some countries will block Twitter usage altogether. And in other countries, the use of geolocated tweets is much less popular, either due to popularity of applications that do not record location by default or a greater cultural awareness of privacy issues relating to revealing your location when you tweet.


Above: Twitter activity in central Edinburgh, proving once and for all that the East End is a cooler place than the West End.

From the Mapbox blog. Found via Twitter, appropriately. Some of the background data is © OpenStreetMap contributors, and the map design and technology is © Mapbox.

Book Review: London Buildings – An Architectural Tour


This book, which features great examples of London building architecture, is itself distinctively designed and immaculately presented. It’s been out for a couple of years now, however I was recommended it when purchasing another book recently on Amazon, as an impulse purchase, it’s an excellent find.

The book was authored by Hannah Dipper and Robin Farquhar of People will always need plates and is based on their heavily stylised interpretation of the buildings featured.

Each building featured in the book – there are around 45 – gets a two page spread, always in the same format – the building shown in white with clean strokes of detail in black, and a distinctive, single tone of colour for the sky. A small inset box includes the buliding name, architects, age and 100 words. That’s it.

The book doesn’t just feature the modern Brutalist London landscape (e.g. Trellick Tower), and the latest modern skyscrapers (e.g. the Gherkin) it also includes such older gems as Butler’s Wharf and the Dulwich Picture Gallery. These two are treated to the wonderful, minimalistic sketch style, with just the two colours allowing the design detail of the building itself to take centre stage.

On Amazon: London Buildings: An Architectural Tour, currently for £9.99. Published by Batsford, an imprint of Anova Books.

Image from the London Design Guide website.

OpenStreetMappers of London


I contributed a number of graphics to LONDON: The Information Capital, a book co-written by Dr James Cheshire, also of UCL Geography. Two of my graphics that made it into the book were based on data from OpenStreetMap, a huge dataset of spatial data throughout the world. One of the graphics, featured in this post, forms one of the chapter intro pages, and colours all the roads, streets and paths in the Greater London Authority area (around 160,000 “ways” which are discrete sections of road/path) according to the person who most recently updated them. Over 1500 indivdual users helped create and refine the map, and all are featured here. I was pleased to discover I was the 21st most prolific, with 1695 ways most recently modified by myself at the time that the graphic was produced.

The more active users will typically have areas around home and work which they intensively map, plus other, smaller areas such as contributions made during a mapping party or other social event organised by/for the London OSM community. Here’s an example filtering for just one user:


Putting the users together reveals a patchwork of key authors and more minor contributors, together forming a comprehensive map of the city. Detail levels vary, partly as the fabric of the city varies from area to area, but also as some contributors will be careful to map every path and alleyway, while others will concentrate on the driveable road network.


The data was obtained from a local copy of the OpenStreetMap database, for Great Britain, that I maintain for various pieces of work including OpenOrienteeringMap. You can obtain the data files from GeoFabrik (this link is to their new London-only version). The data was captured in early February 2014. Newham borough in east London (light blue) shows up particularly prominently because it looks like it had had a bulk update of all roads there by a single user, just before the capture, to indicate which were lit by streetlights (lit=yes).

I used QGIS to assemble the data and applied the temp-c colour ramp, classifying across all the contributors – I then changed the ones which were assigned a white colour, to green. The colours used in the book are slightly different as some additional editing took place after I handed the graphic over. The colour ramp is relatively coarse, so multiple users will have the same colour assigned to them. The very long tail of OSM contributions (where only a small number of people make the great majority of edits) mean that this still means that most major contributors have a unique colour assigned to them.

View larger version.


Note that these files actually are for an area that is slightly larger than the Greater London Authority extent – a buffer from Ordnance Survey Open Data Boundary-Line is used to mask out the non-GLA areas.

If you like this thing, it’s worth noting that Eric Fischer independently produced a similar graphic last year, for the whole world. (Interactive version).

Tube Tongues – The Ward Edition


If you are a Londoner but felt that Tube Tongues passed you by, maybe because you live in south-east London or another part of the city that doesn’t have a tube station nearby, then here’s a special version of Tube Tongues for you. Like the original, it maps the most popularly spoken language after English (based on 2011 Census aggregate tables released by the ONS, via NOMIS) but instead of examining the population living near each tube station, it looks at the population of each ward in London. There are 630* of these, with a typical population of around 10000. I’ve mapped the language as a circle lying in the geographic centroid of each ward. This is a similar technique to what I used for my local election “Political Colour” maps of London.

A few new languages appear, as the “second language” (after English) in particular wards: Swedish, Albanian and Hebrew. Other languages, which were previously represented by a single tube station, become more prominent – Korean around New Malden, German-speaking people around Richmond, Nepalese speakers in Woolwich, Yiddish in the wards near Stamford Hill and Yoruba in Thamesmead. Looking at the lists of all languages spoken by >1% of people in each ward, Swahili makes it on to a list for the first time – in Loxford ward (and some others) in east London. You can see the lists as a popup, by clicking on a ward circle. As before, the area of the circles corresponds to the percentage of people speaking a language in a particular ward. The very small circles in outer south-east London don’t indicate a lack of people – rather that virtually everyone there speaks English as their primary language.

English remains the most popularly spoken language in every ward, right across London. Indeed, there are only a three wards, all in north-west London, where it doesn’t have an absolute majority (50%). London may seem very multilingual, based on a map like this, but actually it is very much still Europe’s English-speaking capital. See the graphic below, which shows the equivalent sizes the circles are for English speakers, or click the “Show/hide English” button, on the interactive map.

Here’s the interactive map. There’s also a ward version of Working Lines.

* I’ve ignored the tiny City of London ones except for Cripplegate, which contains the Barbican Estate.

Background map uses data which is copyright OpenStreetMap contributors. Language data from the ONS (2011 Census).


DataShine: Local Area Rescaling & Data Download

Cross-posted from the Datashine Blog.

DataShine Census has two new features – local area rescaling and data download. The features were launched at the UK Data Service‘s Census Research User Conference, last week at the Royal Statistical Society.

Local Area Rescaling

This helps draw out demographic versions in the current view. You may be in a region where a particular demographic has very low (or high) values compared to the national average, but because the colour breakout is based on the national average, local variation may not be shown clearly. Clicking on the “Rescale for current view” button on the key, will recolour for the current view.

For example, the popularity of London’s underground network with its large population, means that, for other cities with metros or trams, their usage is harder to pick out. So, in Birmingham, the Midland Metro can be hard to spot (interactive version):


Upon rescaling, just the local results are used when calculating the average and standard deviation, allowing usage variations along the line to be more clearly seen:


As another example, rescaling can help “smooth” the colours for measures which have a nationally very small count, but locally high numbers – it can remove the “speckle” effect caused by single counts, and help focus on genuinely high values within a small area.

Hebrew speakers in Stamford Hill, north-east London (interactive version):


Upon rescaling, a truer indication of the shape of the core Hebrew-speaking community there can be seen:


Occasionally, the local average/standard deviation values will mean that the colour breakout (or “binning”) adopts a different strategy. This may actually make the local view worse, not better – so click “Reset” to restore the normal colour breakout. Planning/zooming the map will retain the current colour breakout. PDFs created of the current view also include the rescaled colours.

Data Download

On clicking the new “Data” button on the bottom toolbar, you can now download a CSV file containing the census data used in the current view. Like the local area rescaling functionality, this data download includes all output areas (or wards, if zoomed out) in your current view. This file includes geography codes, so can be combined with the relevant geographical shapefiles to recreate views in GIS software such as QGIS.

Next on the DataShine project, we are looking to integrate further datasets – either aggregating certain census ones or including non-census ones such as IMD and IDACI deprivation measures, or pollution.

Working Lines


As a followup to Tube Tongues I’ve published Working Lines which is exactly the same concept, except it looks at the occupation statistics from the 2011 census, and shows the most popular occupation by tube station. Again, lots of spatial clustering of results, and some interesting trends come out – for example, the prevalence of teachers in Zones 3-4, that there is a stop on the central line in north-east London which serves a lot of taxi drivers, and that bodyguards really are a big business for serving the rich and famous around Knightsbridge.

The northern line (above) stands out as one that serves a community of artists (to the north) and less excitingly a community of business administrators (to the south). Tottenham/Seven Sisters has a predominance of cleaners, and unsurprisingly perhaps plenty of travel agents live near Heathrow. I never knew that the western branch of the central line, towards West Ruislip, was so popular with construction workers. Etc etc.

Only the actively working population is included, rather than the full population of each area. This makes the numbers included in each buffer smaller, so I’ve upped the lower limit to the greater of 3% and 30 people, to cut down on small-number noise and minimise the effect of any statistical record swapping.