Category Archives: OpenLayers

Rise of the Colourful Circles – Election 2010 Visualisation

election

I’ve fixed and tidied up a visualisation I created back in 2010, which showed the results of that year’s General Election in the UK. Newer versions of OpenLayers had broken it (specifically the use of addUniqueValueRule with a custom context resulted in no circles appearing) and also the UI looked rather rudimentary. Now it has rounded corners, transparency, more spacing and a prettier font!

Although it was my first “coloured circles” visualisation (the Bike Share Map followed on from it a few months later when London’s system launched) it was my most sophisticated, with the circles having different coloured areas and borders, and changing in size – plus a view where the colour itself is calculated from the numeric values – select the “Constituency Colour” from the first pop-down.

The key benefit of the circles other the traditional “colour in the constituency” election map is that sparsely populated rural areas did not dominate the map. It also means that, when viewing the results from individual parties, that each pixel of each colour represents exactly the same number of votes – whether in central London or the Highlands of Scotland.

The background map is not great at all – a mess of greys and names. At the time, I was strictly keeping colour out of it, so that the only colour was the data being visualised. The early Bike Share Map also had the uninspiring background, with a dark grey river flowing past lighter grey lands. These days I’ve relented – a small amount of colour is OK, as long as the shades are pastel and appropriate, and the key data’s colours are vibrant.

You can see the map here.

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Rank Clocks and Maps: Spatiotemporal Visualisation of Ordered Datasets

Rank Clocks are a type of visualisation invented by Prof Michael Batty here at UCL CASA. They are time-based line charts, wrapped around a clockface – with the start date at the top, wrapping around clockwise to the end date. The lines on the clock show the change in ranking of the items being visualised. By effectively wrapping a line chart around itself, certain patterns, that would be otherwise hard to spot, become clearer.

Starting from Prof Batty’s Rank Clocks application (written in VB), I created a web version that has a subset of the application’s features, but also includes a map, allowing both temporal rank changes, and location, to be shown. A future enhancement would also be to show the change in location with time as well (an example would be how football clubs have moved around in London over the years and how their relative rank in the leagues has also varied) but for now each item in the dataset has just a single point location that remains constant with time.

Live Rank Clock site here.

The “classic” Rank Clock is of New York skyscrapers – looking at the clock allows bursts of skyscraper development to be easily spotted, and as New Yorkers have been building skyscrapers for over a hundred years, and have many of them, it is a rich dataset. I have curated a London equivalent from various sources including Wikipedia. It includes the many residential towerblocks of the 1960s/1970s, many now knocked down, but is not quite the same as New York’s.

The website is written in Javascript, using OpenLayers both for the map (with OpenStreetMap background) and for the rank clock itself. For the rank clock, I am doing some basic trigonmetry to calculate the coordinates needed to show the lines and converting from polar coordinates to “native” screen coordinates. This is a novel but not particularly efficient use of OpenLayers, but I used it as I am quite familiar with using OpenLayers, particularly for showing lines as vectors, rather than using a Javascript vector-based charting API which would be the more obvious choice.

My interpretation of the Rank Clock concept has plenty of flaws – in particular, data can often be easily obscured, and spotting patterns in noisy (frequently changing rank) data is difficult. It’s difficult even to select lines (to see their caption) if other lines are nearby and overlaying them. Nonetheless, it can provide an unusual way of looking at some interesting datasets.

For one of the datasets in the sample website (US baby names) I have repurposed the map to effectively show a 2D graph indicating beginning and ending (in time) positions of the names – so here OpenLayers is being used to show two “maps” – but neither are actually maps.

I’ve also linked into the Google Earth browser plugin (installation maybe be required), replacing each dot on the OpenStreetMap map, with a column of varying height (and colour) based on the initial rank, with an extent appropriate to the data set. Google Earth can be refreshed by supplying new KML information – and it turns out that OpenLayers has a rather nice KML conversion and export feature for any geometry in it, which allows Google Earth to be driven in this way. This is done when clicking on a Rank Clock line, allowing the equivalent feature in Google Earth to be redrawn with a thicker border. Unfortuantely events cannot be captured from Google Earth and back into the OpenLayers map, so clicking on a pillar in the former will not highlight the corresponding Rank Clock line in the latter. Still, it’s a nice way of linking spatialtemporal information and then visualising it in 3D.

I carried this work out quite a while ago, but haven’t mentioned it to now, as it’s not complete. There are only a limited number of datasets available, and plenty more features could be added – and the navigation and interaction improved significantly. Please bear this in mind when viewing the live site.

There are a few “toy” features already though – you can invert the rank clock (normally the top-ranked items are in the middle of the circle and so are hard to see), change the metric the colour is showing, and filter and relayer.

The three rank clocks shown here are showing: TOP – Changes in population of the London Boroughs of Newham and Tower Hamlets, and the City of London, over 150 years. The City of London line spirals outwards, showing its drop in population (and so rank). Tower Hamlets also shows a big drop in rank during WWII, but has started to increase again recently. Westminster’s population rank has steadily increased, until WWII – but again its rank has also more recently increased. MIDDLE – Tall buildings in London, coloured by year they were built. The oldest (red) buildings have been selected and show in Google Earth, showing that such buildings were entirely in the centre and west of London. BOTTOM – US company revenue. The San-Francisco-headquartered companies are selected on the map and correspondingly highlighted on the rank clock, showing that only one was founded before the 1970s – IBM – and a general spiralling inwards as Silicon Valley grows.

Live Rank Clock site here.

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Review: OpenLayers – The Book

OpenLayers 2.10 Beginner’s GuideErik Hazard, Packt Publishing

The OpenLayers 2.10 Beginner’s Guide (buy: Amazon) is a guidebook for a technology that is relatively unknown in the broader technology world – OpenLayers being a JavaScript-based online mapping framework, the open-source equivalent of the popular Google Maps API. As mapping and open source software have increasingly become mainstream over the last couple of years, such a guide has appeared at just the right time.

As with many open source projects, documentation and startup guides are freely available online, but their quality and completeness often is very variable. A paper-based guidebook is still the best way to get to grips with the complete capabilities of a complex framework like OpenLayers – but with the danger that it is likely to date quickly as the project evolves. The author is particularly brave in referring to the current version – 2.10 – in the book title, as any subsequent release (2.11 is probably just around the corner) will appear to age the book, while actually it is likely the vast majority of its content will remain relevant with the new version.

The book serves both as an introduction for beginners to JavaScript and/or online mapping, while also acting as a reference that means it can remain in the development bookshelf for even more advanced developers. There is not a standalone reference section – but then the automatically-generated JavaScript documentation online perhaps serves best for this. However the detail of the chapters mean they effectively can act as a near complete reference.

OpenLayers does have a notably steeper learning curve than (for example) the Google Maps API – although it is ultimately more powerful. A physical, complete guide like this does therefore have a definite benefit to a developer aiming to produce online mapping applications based on open source technology.

The book, slightly surprisingly, introduces some fairly weighty programmatic detail right at the beginning – in Chapter 1. For example, I wasn’t expecting a discussion of Object Orientated Programming, and defining objects, instances and classes/subclasses, so early on in the book. It could be argued that these are important concepts to learn early, o gain a good understanding of a powerful API. I do still think that these would be likely to intimidate a genuine beginner, who just wants to quickly create a map.

The second chapter covers Firebug in depth. While again I was not expecting this so early in the book, I can understand why it is introduced at this stage. It is easy to make typos when writing JavaScript, and a working knowledge of Firebug stops painful debugging that might be necessary for someone just starting out with JavaScript development. But it is possible to use OpenLayers and dip into its rich API without knowing about Firebug, and such detail might serve to intimidate a novice, who would get buried in the detail – hence why I was surprised to see this inclusion so early.

It is interesting that OpenStreetMap (OSM) was not introduced until a third of the way into the book. Before, complex concepts such as map projections and coordinate transforms are used. You really don’t need to know these to use OpenStreetMap! OpenLayers is of course used for so much more than just showing OSM maps – at a recent academic conference I was impressed by the number of speakers that showed complex web applications using OpenLayers. But the “simple case” for OpenLayers is pulling in Google Maps or OpenStreetMap map tiles, and putting pins on them – “mashing up” data, without worrying about projections, ESPG numbers, JavaScript debugging and WMS parameters – all of which you’ll meet early on in the book. Only on page 124 is a simple mashup (Google + WMS + vector editing) introduced, with a more complex example application dissected in the very final chapter.

The vector layers – which in my opinion are the most interesting and powerful part of OpenLayers are introduced all the way back in chapter 9. Vector styling – advanced but powerful – is in the chapter after that. There are many complex method calls for vector layers, and so many of these do deserve to be introduced only late on – but introducing a subset of vector capabilities much earlier in the book would have been of benefit.

There are several typos and minor misspellings in the book, including in a couple of code examples, detracted slightly from the clarity of the book.

The OpenLayers 2.10 Beginners Guide is fantastic as a reference – it goes into great detail and is so very useful for advanced users of the OpenLayers API. But it forsakes smoothing the OpenLayers learning curve in favour of a solid ground-working of the detail – great if this is your career, not so great if you just want to download OpenLayers and create a simple map. If you are prepared to spend the time with the detail, then you will emerge an OpenLayers expert!

The book’s accompanying website includes Chapter 9 as a sample. You can also buy the book – OpenLayers 2.10 Beginner’s Guide – from Amazon.

Heatmaps are Simple with HTML 5 and Canvas

Cross-posted from my orienteering blog.

As a Saturday-lunchtime project, I have created a heatmap of where the 2700-odd geolocated orienteering races have been held in Great Britain in the last two years.

The heatmap quickly shows clusters around the main urban areas, where the population sizes supply participation for many local events being put on. Another major bright-spot is the Lake District extract on the right – which contains a large amount of high quality terrain for events. Other areas, such as the Cotswolds NW of Oxford, seem to be somewhat underused.

If you have a browser than can handle the HTML5 Canvas tag (i.e. not Internet Explorer!) you can view the heatmap here. Zoom into your local town or city to see if events have been held there – when zooming in, you’ll need to adjust the two sliders most of the way to the right, so that individual events show up. With the individual settings, a single, isolated event will have very little impact on the heatmap.

The heatmap was possible thanks to the excellent Heatmap library produced for OpenLayers by Bjoern Hoehrmann. The map is powered by OpenLayers, with an OpenStreetMap basemap. I’ve used a custom colour ramp, based on one supplied by Colorbrewer. The custom map adornments are supplied by MapBox.

Creating a heatmap like this is very easily, with just a few lines of Javascript needed to add objects in. The Heatmap library does the rest. See Bjoern’s example for the documentation.

The First Million London Bike Share Journeys

Thanks to a FOI request from Adrian Short, Transport for London have recently released to their developers area details of 1.4 million bike share journeys. The data is believed to include all the journeys between 30 July 2010 and 3 November 2010, except those starting between midnight and 6am.

I’ve created a map which visualises these journeys – select a docking station and a time, and it will show the journeys that start/end at that dock, depending on the options chosen.

You can see the map here. On launching the site, an initial docking station – one outside Waterloo station – is selected, and an “interesting” timeframe is chosen – the morning of 4 October, which was a day impacted by a tube strike.

Heavy usage along the Broad Walk through Kensington Gardens, particularly at weekends:

The predominant flows from a docking station near King’s Cross station, in weekday mornings, are outwards (red lines), particularly south towards the river. Only a few inbound journeys happen (blue lines):

The reverse is true in weekday evenings, as commuters head back to the stations:

The map bears a resemblance to my live Barclays Cycle Hire scheme status map, as I’m reusing a lot of the same code and graphics.

More Circles on a Map – Orienteering Fixtures

Five years ago, I created a mashup of forthcoming orienteering fixtures in Great Britain, as listed by the sport’s national governing body, British Orienteering, on its website. It was based on the Google Maps v2 API, and a regular scraping of the HTML on their website, and was a set of pins on a map, coloured by the number of weeks to the event. On clicking a pin, you got a popup balloon with details of the event, and a link to the organising club’s website. A postcode locator, based on data from the NPEMap project, was added, so you could focus on events in your local area. You could also filter out far away events.

A couple of years later, British Orienteering’s web developers added their own map to their website – Google Maps v2 API based, with pins coloured by the number of weeks to the event, and a popup balloon, a postcode search and distance filter etc etc… The Unique Selling Point of my fixtures map was lost.

So, when a rewrite of British Orienteering’s website just before Christmas broke my map, I took the opportunity to rewrite it, as a vacation project, using the technologies I’ve been using a lot in 2010 – namely OpenLayers, OpenStreetMap, OS OpenData and coloured vector circles. The map is bigger, brighter, and hopefully more useable than the official map and my previous version.

You can see the new map here – with a mass of dots representing forthcoming fixtures, and circles surrounding the “home” postcode, backed by OpenStreetMap, with the postcode locator based on CodePoint Open from Ordnance Survey OpenData. Only the locator uses a database, the rest of the webpage is constructed on-the-fly from a webpage regularly copied from the British Orienteering website.

Not Scarborough…

The map remains subject to the quality of the data entered on the corresponding list – there is some limited tidying up of the data, but it’s difficult to correct grid references that result in events being in the sea – there’s currently one in the Irish Sea, as the event registrant entered “GR” as the grid reference letters, and this just so happens to be the location of the GR myriad. There is still work to be done on my new map, such as spotting obvious errors like this, guessing locations where a grid reference isn’t supplied, and perhaps including Northern Ireland’s events.

Incidentally, my original orienteering web map, which inspired my fixtures map, was one showing orienteering maps, it was written way back in August 2004, using a Flash mapping package by Map Bureau, with dots superimposed on top of a map pinched from Wikipedia. We’ve come a long way.

The Political Colour of Great Britain

Following on my UK General Elections 2010 Swings map, where circles represent each constituency, with the sizes and colour describing the metrics, I have used a technique that James has been studying, to combine the vote proportions together to produce a single coloured dot for each constituency. The more blue the dot, the higher the proportion of the vote was Conservative. Similarly, red for Labour and Green for the Lib Dems and the other parties and independents.

A purple constituency represents a Labour/Conservative marginal, as blue+red = purple. There are many of these in the Midlands. Similarly, orange areas indicate likely Labour/Lib Dem (or SNP/PC etc) marginals, such as in South Wales, and turquoise areas indicate Conservative/Lib Dem (& other) margins, there are many of these in SW England. Grey dots show three-way marginals, e.g. Hampstead in London.

Because I am using just three colours, to represent three political groupings, the visualisation does not show the variation in results between the Liberal Democrats and the nationalist parties – the SNP and PC in Scotland and Wales respectively.

Some interesting patterns are revealed – the ring of blue/purple around the red/orange centres of London and Birmingham – with no such corresponding ring around Manchester or Newcastle. Scotland’s lack of blue. The straightforward Labour/Conservative split in NW England. East London isn’t quite as safe-Labour as you might think – with the exception of East Ham standing out in bright red.

The benefit of this visualisation is that every vote is included in it – on a regular election map, if one party just fails to win, then an election map won’t show them at all on it. Here, every vote influences the colour of the map. Each circle represents roughly the same number of people – the populations of constituencies are fairly even, with some notable exceptions such as the Isle of Wight (very populated) and the Scottish Islands (very few people.)

You can see the colour map by going to the election visualisation and choosing “Constituency Colour” from the “Other graphs” drop-down. James has produced a regular choropleth map version, which shows the green (SNP/LD) of northern Scotland and blue (Conservative) of southern England strikingly well.

Tube Stations in London – Visualisation

I was inspired by seeing this map and associated article on the New York Times website, linked from Going Underground, to create a similar mashup/visualisation of entry/exit volumes from the 300-odd tube stations in London. On their website, Transport for London provide the metrics for entries/exits from the stations, between 2003 and 2008, broken up into rush-hour, regular and weekend travel.

Each circle’s area is directly proportional to the flow numbers for that station (click on the circle to see the numbers.) The circles are rescaled between the first metric (total flows) and the rest, so direct comparison of metrics is possible except between the first and others, Blue circles represent an increase in flow and red a decrease.

If the mass of circles are obscuring each other, zoom in!

You can try it out here.

Some technical notes:

The background map is a custom render of OpenStreetMap data, with the tube lines highlighted in their traditional colour – it doesn’t always look quite “right” when you zoom in, due to the way the lines are tagged in my own copy of the OpenStreetMap database. The stations are even harder to disambiguate, so I’m using a free source from Wikimedia Commons, this means they don’t always line up.

Because your browser gets a copy of all the flow data when you load the page (yes I’ve heard of AJAX) it does run a little slowly in Internet Explorer, particularly the slider bars – these allow you to “drag” through the range of metrics or years.

Manchester Map Mashup

I’ve created a mashup of lots of maps of Manchester as a proof-of-concept of how easy it is to mashup using OpenLayers. It’s not particularly pretty but does involve lots of maps.

See it here.

The layers are:

  • OpenStreetMap
  • Ordnance Survey Street View
  • Ordnance Survey 1:25000 First Series (1959)
  • Ordnance Survey New Popular Edition (1948)
  • Marr Map of Housing Conditions (1904)
  • Swire Map of Manchester (1824)

The first four maps are all hosted on OpenStreetMap servers.
The Swire map also contains an inset, dated 1650!

Accuracy vs Completeness: OSM vs Meridian 2

[Updated x2] Yesterday’s Ordnance Survey OpenData launch has provided the OpenStreetMap community with a potentially rich set of data to use to complete the map of Great Britain. OpenStreetMap’s accuracy and detail is generally excellent, however a problem which is (very arguably) more important than either accuracy or detail, in a map is that some parts of the country are substantially incomplete.

It’s not that the data quality is poor, it’s that someone with a GPS (or a satellite photo) has never been to that part of the country to gather the data in the first place. There are still significant parts of Scotland and Northern England which have many missing roads. The NPE (out-of-copyright) maps have been useful in starting to fill out these sections, but there’s always going to be a roads missing from a 60-year-old (or older) map.

So, the OS datasets could be very useful. Perhaps the most interesting of the datasets is Meridian 2, it is a vector dataset covering the whole country. One thing that needs to be watched out for though is that Meridian (which is a “complete” dataset of the country) is relatively inaccurate Pixellation or resolution isn’t a problem, it being vector based – but data is quite simplified.

I’ve built a mashup which allows direct comparision of the Meridian and OSM data for Great Britain. I’ve added in most of the available layer files that come with the Meridian package that has been released as part of the OS OpenData initative. The only two areal ones I’ve added are for woodland areas and lakes – everything is linear. I’ve added in labels for the roads and rivers, but no boundaries or point features, at this stage.

You can access the mashup here. (N.B. Not tested in IE so will probably break horribly in it.) Zooming in reveals the relative coarseness of the Meridian data – although crucially it is “substantially” complete for all but the smallest of roads, for the whole of the UK – not just for the major cities where the OSM contributors mostly live!

Completeness
In the pictures below, the “solid”, thinner roads are Meridian and the fatter roads with “borders” are OSM.

Spot the missing roads in Meridian around Leytonstone in East London [Update 1 – Some sections of motorway are missing from my rendering but are present in the data – it is possible this problem extends to smaller roads too so take these screenshots with a pinch of salt]:

…but go further out of London, and it doesn’t look so good for OSM:

Interestingly, the Park Estate in central Nottingham is missing entirely from Meridian:

The Park Estate is a private estate and the roads are not maintained by the council – this might have something to do with it. I’ll be running around the Park Estate next weekend.

Accuracy
[Update 2 – Meridian is not intended to be used at scales larger than 1:50000, as per its documentation, so I shouldn’t really be comparing it with OSM which generally is based on data recorded at larger scales. So, bear in mind these screenshots are all larger than 1:50000 scale.] It’s difficult to authoritatively judge the relative accuracies of the two datasets without getting out on the streets or looking at aerial imagery – but you can infer a basic measure of accuracy by looking at how roads “wiggle” – or, in the case of the Mayfair squares below, how Meridian converges the square to a point:

Detail
A little unfair to compare the two here, as Meridian 2 was always meant to be a medium-scale dataset, whereas OSM can be all things to all people!

The tiles that make up the imagery are generated on demand (and cached for subsequent use) so may run slowly. You’ll need to zoom in quite a long way before all the features get added to the map. Use the slider on the top left to fade between the OSM and Meridian layers.

The images are derived from Ordnance Survey data © Crown copyright and database right 2010 and OpenStreetMap data which is CC-By-SA OSM and contributors.