Category Archives: OpenStreetMap

Someone’s Mouse Slipped

…when they were plotting this road in East London for Google Maps:

A quick look at the Google Maps aerial imagery confirms there’s something odd about that road kink:

Never fear, OSM is here:

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Vienna: State of the Map EU

So – I was at the State of the Map EU (SotM) conference in Vienna last weekend – a European-focused conference on the OpenStreetMap project. I travelled with my colleague Steven Gray and presented some screenshots from the GEMMA project I am currently working on at UCL CASA – more about that in a later post. The two of us, and London OpenStreetMappers Shaun and Tom, stayed at the shiny new Wombat Nachtmarkt hostel which was convieneintly a few minutes walk from the venue at Vienna University of Technology (TU Wien). I was impressed that, on walking onto the university campus, my phone connected seamlessly to the Eduroam wireless network, based on my UCL credentials – a feat that was not managed in recent trips to more local academic campuses in Manchester and Imperial.

I was impressed with the number of people at the conference – over 200, which was larger than the global SotM conference I was at in Amsterdam two years ago. According to the stats, 2/3rds of people there were from the German-speaking diaspora (Germany, Austria and Switzerland) which demonstrated the clear demand for a SotM conference based here.

I mainly followed the “Tech” track at the conference. I was particularly interested to know about Mapnik Metawriters, which I’m looking to incorporate into some forthcoming Mapnik-based work. You know when you click on Google Map POI icons (not pins) and you get a tooltip with the name of what you’ve clicked on? It’s similar to that. Another highlight included Andy Allan with a tour of custom cartography of OpenStreetMap data. Andy’s cartographic-focused talks are always a visual feast. Unfortunately my own talk clashed, but I managed to make a quick exit after mine and caught the last bit of his.

Another interesting talk was ESRI’s launch of version 2 of their OpenStreetMap editor for ArcGIS – OSMEditor. Of course, you still have to have a copy of ArcGIS in order to be able to use a plugin – so the non-academic, non-commercial audience is unlikely to be using it. I was slightly surprised the presenter didn’t mention the $100 non-commercial licence that is now available for ArcGIS. The $0 price-point for Quantum GIS (which also has an OSM editor plugin) is still going to be unbeatable, but ESRI is certainly going in the right direction. Their engagement with OSM is not something I would have suspected a couple of years ago, it’s great to see them sponsoring and presenting at a conference like this. Of course, having the OSM layer a click away in ArcGIS as a background layer is a good win for them too. And they even let us call them “esri” these days! 🙂

Muki Haklay gave an overview of his team’s completeness analysis for the UK OSM dataset over the years. We used to say we “are good enough”. Now we can say that, subject to qualifications, we are “as good as” some traditional datasets. There was also some similar research presented by Heidelberg University, which used hexagonal cartograms, which was an interesting change from grid squares. I should also mention Steve Coast’s keynote, which was a frank statement of the current state of play of the project – good in many places, but problems with the Australian community feeling disengaged and looking to split from the project were clearly top of his mind.

It was great to meet face-to-face with some major figures in the community – notably Frederick Ramm of GeoFabrik. I managed to sit beside him for half an hour at the conference dinner without twigging who he was. Frederick is one of the authors of the OpenStreetMap book that I reviewed – one of my comments was used as a quotation in the book’s advertising at the conference!

Henk Hoff from the OpenStreetMap Foundation was in fine form, with one of his “poster auctions” at the end of the conference. He also announced the winner of the free trip to the “father” SotM conference in Denver in September being Gregory Marler. Gregory won with his Rebecca Black-esque recording “Fly me to SotM” (I hope he doesn’t mind me saying that!)

The social side of the conference was excellent. Plenty of breaks for networking, and a conference dinner on the Friday night. This involved everyone getting a couple of specially hired 1920s wooden trams (or “Bims” after the sound their bells make) to a suburb of Vienna – via the grand ring-road, past the various palaces and other grand buildings – whereupon we took over most of a restaurant for an Austrian feast of Wiener schnitzel, meat loaf, sauerkraut, picked cucumber, and a dessert of apple strudel. A few resturant-brewery combinations were also visited during the trip – along with some most refreshing lagers, served in proper glasses with handles that make a lovely “clonk”. Vienna was very warm indeed, with a thunderstorm on the first night. It was also eerily quiet – the city is quite grand and spaced out, plus maybe many of the locals were on holiday to the mountains. Certainly the people we met were friendly. I should mention specially the conference organisers, which were flawless and ensured everyone was in the right place at the right time! The organisation of the conference and social events appear to go off without a hitch.

It was a great trip to see what’s going on with the OpenStreetMap development community, present some of our own work at CASA, and explore Vienna.

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

A Historical Comparison of OpenStreetMap’s Completeness in Britain

Dr Muki Haklay,UCL CEGE, has been carrying out some quantitative research into OpenStreetMap’s coverage in the UK, comparing road lengths in each square kilometre, with those in a definitive national dataset, Ordnance Survey Meridian 2. He’s updated his findings every few months, from March 2008 until March this year. Some interesting research findings have been found, such as a potential correlation between an area’s affluence and the map’s completeness, a possible reflection of a contributor demographic. On his suggestion I’ve taken his dataset and overlaid the red/blue under/overcompleteness maps on OpenStreetMap (or Ordnance Survey StreetView) itself, allowing the specific towns and villages that are missing the OSM love, to be identified.

The mashup can be viewed here.

These days, OpenStreetMap’s coverage is pretty good -often exceeding Meridian’s, as service roads, private roads and alleys, that don’t exist on Meridian 2 are added in. There’s still (as of March 2011) some significant holes though, particularly in parts of Wales, the North East and East Anglia.

Note the first four maps only cover England. There is an interesting artefact in the first one – a square around London can clearly be seen, corresponding to the extent of aerial imagery, in that area, that was available via a special agreement with Yahoo for tracing. Outside of that area, only 50-year-old (out of copyright) maps and contributor GPS traces were available. Since May last year, the Ordnance Survey OpenData release, and Microsoft Bing Aerial imagery, which became available at roughly the same time, has significantly accelerated work on the map. I presented on the diverse sources of data at the Society of Cartographers annual conference last year, you can see the slides here.

ITO World’s OS Locator is just one of a number of tools that the OpenStreetMap contributor community in the UK is using to “complete” the map, moving towards the goal of a comprehensive free database of the UK’s (and world’s) streets.

Hackney: “Complete” on OpenStreetMap

ITO‘s OSM Analysis table is updated daily, showing the number of roads in each district/borough in the UK that are in the OS Locator* dataset that are missing in OpenStreetMap. There is an accompanying choropleth map (you need to login to seet it) showing coverage across the UK. Currently southern England and Scotland’s central belt are looking pretty good, while Wales and parts of northern England still have quite some way to go.

As well as the map and summary table, ITO produce a map showing the approximate location of each missing (or misspelled) road, as a rectangular bounding box. This makes finding the missing roads quite easy. Groundwork is needed to check signposts and confirm the names. Often, discrepancies arise simply from OpenStreetMap not having apostrophes for the street name, and the Ordnance Survey having them, or vice versa. The signposts normally provide the definitive answer, but in quite a few cases, the sign at one end of the street will have an apostrophe and the other end will be missing it – or the names attached to houses will differ from the nearby street signs.

Hackney was around 94% complete a few weeks ago, with around 80 errors flagged up. It’s a small borough (in size, if not population) so I reckoned it was possible to bike around all 80 locations in a day – with a suitable route that would hopefully be a good answer to the Travelling Salesman Problem.

In the end, I managed it in a couple of afternoons. I found quite quickly that in most cases, the street was already in OpenStreetMap but just misnamed or unnamed. In around 5-10% of cases, the Ordnance Survey was wrong, and street signs on the ground suggested either that OSM was correct, or that both sources was wrong. As the OS was in general right, I only stopped to note where this wasn’t the case – so, I was able to cover a lot of ground quite quickly. I still ended up having to cycle nearly 100km within a borough that is roughly 4-5km across.

Perhaps the most interesting part of the exercise is it allowed me to visit all parts of the borough, including places I had never been to before despite having lived there for several years. I discovered that Stoke Newington Church Road is indeed massively gentrified out of all proportion to the surrounding area. I found unexpected gems like The Mothers Square (no apostrophe) and that Hackney Downs is quite a pleasant park. I also found the huge Pembury Estate. This was where one of the Ordnance Survey’s mistakes was – the streets were named in the wrong order. Perhaps the original surveyor didn’t like to hang around.

Of course no map can ever be 100% “complete”. Even with a perfect match to OS Locator, the latter may be out of date, or be missing roads due to missed records. The map may be “complete” in terms of roads but other detail still needs to be added.

Anyway Hackney is now 100% complete with respect to OS Locator, making it the 12th such district in the UK. Another 400-odd to go…

* OS Locator is one of the products in Ordnance Survey’s OpenData release.

Review: OpenStreetMap – Using and Enhancing the Free Map of the World

OpenStreetMap, the free wiki world map, is starting to come of age. The project is now six years old, and is gradually becoming noticed in wider circles, with AOL and Mapquest producing their own versions of the map, support from Google and Microsoft, and an ecosystem of companies set up around commercialising the data. Perhaps the highest profile the project has had recently was when, in the days following Haiti’s huge earthquake in January 2010, the country was swiftly mapped remotely in high-detail by contributors from around the world, becoming a useful tool for disaster relief teams. Meanwhile, the project continues to expand, with the availability of high-resolution Microsoft Bing imagery causing a big jump in the detail being added to the map recently, and new countries and areas continuing to be worked on actively.

A couple of OpenStreetMap guides became available just before Christmas, and I have one of them – OpenStreetMap: Using and Enhancing the Free Map of the World. It’s published by UIT Cambridge and available on Amazon. The authors are Frederick Ramm, Jochen Topf and Steve Chilton. The book is in its first English edition – Ramm/Topf wrote the previous German editions of the book, while Steve Chilton has translated the work and updated with the latest developments. The authors are a real authority – Ramm/Topf are well known in the German OpenStreetMap community, running a company Geofabrik which builds on OSM, while Steve Chilton (pictured below at an OpenStreetMap conference) is a professional cartographer who has designed and maintained the “standard” OpenStreetMap map you see at http://osm.org/.

The book has a dual purpose – to act as both a guide for using and getting the most out of OpenStreetMap data, and contributing to it. It runs to over 300 pages and is split into four sections. The first section is an introduction, it outlines the basic structure of the project and the community behind OpenStreetMap. This is followed by the longest section in the book, which details how to contribute to the project, from wandering around your local street to using one of the available editors. It includes a detailed guide to Potlatch, the online editor which many people will use when starting out with contributing to the project. It also introduces several other editors, which is good in terms of balance, although new users do not necessarily need to learn more than one.

The third section is about taking OpenStreetMap data and creating maps from it. Examples of cartographical style sheets are included and carefully dissected. There is some code here, but it is well annotated so shouldn’t prove too intimidating to read. Finally, for the most advanced users, the fourth chapter details getting down under the hood of the OpenStreetMap architecture and “hacking” the data, including documenting the API calls to OpenStreetMap servers to programmatically get and put information.

One of the most interesting parts of the book is actually one of the appendices, detailing country-specific quirks of the project. One of OpenStreetMap’s greatest strengths is that it is a map and spatial database of the whole world – but individual countries have different available data sources, “tagging” customs for features, and community structures, and the appendix makes for insightful reading. Being a UK contributor of a British-founded project, I find it can be easy to overlook regional differences, so it’s good to understand why some countries have mapping features in a certain way.

I found the book extremely comprehensive. I have been a long-time contributor to the project and have used the data in numerous ways, but I was surprised to learn several new details from the book, such as how to set up banned-turn indicators, and a clear discussion of the licensing.

The book is perhaps most valuable over and above other project resources because it has a consistent editing style and level of detail. The “traditional” way of learning OpenStreetMap has been through the online wiki, but this is prone to varying levels of accuracy and detail depending on the enthusiasm of the authors – so having a single-style book like this is a considerable help in fully appreciating a very diverse project. I did feel that the section on contributing is slightly longer than it needs to be – choosing just one of the editors, such as Potlatch, rather than going into multiple editors in detail, many of which duplicate functionality, would have helped shorten this section. As, I think, people are more likely to read this book to understand how to use the project’s data and resources, rather than become advanced contributors, I would imagine many readers will end up skipping over the whole section. I did also think a history of the project would have been a nice inclusion.

Books on fast changing internet projects such as OpenStreetMap are prone to go out of date quickly. With this in mind, the authors have created a special website, http://www.openstreetmap.info/ which will contain updates to the book as the project continues to evolve. As the English edition has just been published, the book itself however is bang up to date and so stands as a definitive reference.

The website also has a PDF version of one of the chapters in the second section, Mapping Practice. You can also download a copy of the country-specific appendix.

The book succeeds in simultaneously being OpenStreetMap for Dummies, OpenStreetMap: The Missing Manual and the O’Reilly OpenStreetMap book – that is to say, complete beginners, intermediate users and enthusiasts/hackers will all get something out of the book. If you are at all interested in the OpenStreetMap project, even if you don’t intend to contribute to the project but are just curious about what it is or what you can do with it, then I recommend this book. It’s as near-perfect as any book can be about one of the web’s, and the geospatial community’s, most exciting projects. More details on Amazon.

The photo of the author Steve Chilton is by Chris Fleming. Below is an example of advanced OpenStreetMap map cartography for part of Birmingham, CC-By-SA OpenStreetMap and contributors – in particular Andy Robinson.

Boris Bikes Flow Video – Now with Better Curves!

Dr Martin Austwick and I have produced an updated version of the animation of Barclays Cycle Hire bikes on a typical weekday:

Martin has once again done some programming magic to show the River Thames, Hyde Park/Kensington Gardens and Regent’s Park to add context, plus the trails for the bike “motes” are longer, allowing the road network to be picked out more easily – and the network lines remain as faint “ghosting” in the video. The bikes are also more blue! Although the bridges aren’t specifically marked, their locations quickly become obvious from the volume of bikes crossing them.

I’ve redone the routing, to fix a few problems around Trafalgar Square and a couple of other obvious places. As before, the routing is done using OpenStreetMap data and the Routino routing scripts, optimised for bike usage (i.e constant speeds on all road types, obeying one-way roads and taking advantage of marked cycleways.) I’ve tweaked the desireability of road types, so that trunk and primary roads are now only slightly less desirable than quieter routes. The traffic in most parts of central London is so slow that, based on my own observations, such roads are not such a significant deterrent to cycling. As before, I’m assuming the bikes go along the “best” route, I don’t know where they actually went. Hires that start and end at the same point – popular in Hyde Park – are shown with the motes spinning around the point.

I’ve also included road curves this time. This means bikes don’t go in straight lines between junctions. This was particularly noticeable when they cut the corner of the Thames in the last animation! Watch the bikes as they carefully curve around the kinks of West Carriage Drive in Hyde Park, around the graceful arcs of Regent Street and Aldwych and along the Victoria Embankment. (I don’t think there are many other classic curves in the central London area?)

Expand the video to full-screen, and, if your connection can take it, click the HD button to get a higher-quality with even bluer bikes!

The data for the bikes themselves is from Transport for London, with the Thames, parks and the underlying network being faithfully drawn by OpenStreetMap contributors. One of the great advantages of using OSM data – apart from it being easy to access, is it’s often very up-to-date. For example, you can see the kink at the northern end of Blackfriars Bridge, on the animation, where the road bends around the Blackfriars Station redevelopment site.

Flow Animation of Barclays Cycle Hire Bikes

Dr Martin Austwick and I, here at UCL CASA, have been working on an animation of the Barclays Cycle Hire bikes (aka Boris Bikes) in London, based on the historical flow information that was released by Transport for London (TfL) last month.

Taking one of the busiest days of the scheme – the 4th of October last year, a Monday which coincided with a London Underground strike – Martin has created an animation showing pulsing blobs, or motes, representing the bikes, moving through the 18 hours of the day that the data is available for. As each hire is made, the docking station dot flashes red, and and blue trail starts to leave it, heading towards the destination dock which flashes yellow as it receives a bike.

At the rush-hour peaks (08:45 and 17:45) the map becomes a sea of a 1000 blue pulses, many congregating on a number of key routes in London. The few bridges across River Thames can be picked out as intense bars of light, as commuters travel between Waterloo/South Bank and the City/West End. Hyde Park (middle left) and Regents Park (top left) are noticeable from having few docks in their area, and only a few bikes crossing them. The east seems busier than the west, as the City workers typically commute to work earlier and so dominate the scheme on strike day.

Martin’s used Processing, a rich Java graphics library, to create the animation, which has been then output to video. This allows the up-to-1000 bikes to be animated smoothly and effectively.

The bikes are in official Barclays Blue, although if you don’t view the video in HD, they look slightly washed out. Watch the video on the Vimeo website in HD, although you’ll need a fast computer and a broadband connection.

The routing is done based on the OpenStreetMap data for central London. I used Routino to do the routing, producing a routing file for each of the 137,000 possible journeys between docks in London. The routing is directed, meaning the bikes won’t cycle the wrong way down a one-way street. They also generally avoid trunk roads, such as Euston Road, preferring to use the quieter roads and dedicated cycle lanes nearby. Being able to use the new cycling infrastructure in the routing, is one big advantage of using OpenStreetMap.

A disadvantage is where the routing is wrong. For example, access from the Embankment is not shown correctly. Another problem was the reluctance to cross Trafalgar Square in the centre of the city. This meant I had to move a couple of the docking stations slightly. An example of the latter is shown in the picture here. These quirks, and a few others, result in some bikes flying around the animation extremely fast, as the router sends them a mile up in one direction, around a roundabout, and back down in the other direction. The speeds of the bikes are based on the duration information for the journey, which is included in the data, so they start and finish at the right time.

The routing is the “best guess” route, based on the assumption that the majority of cycle users will know the “best” route to take. Casual and multi-stop use will be less accurately shown. Bikes which are returned to the same docking station they started from, are shown “orbiting” the dock for four times, before returning to it.

The work follows on from a recent animation showing the TfL buses in London, by Anil Bawa-Cavia, also here at the UCL Centre for Advanced Spatial Analysis in London.

Wheelmap – The Right Way for the World to Tag OpenStreetMap

I recently came across Wheelmap, a website (and also an associated iPhone app) specifically built to display – and accumulate – wheelchair accessibility information for points of interest (POIs) such as pubs, cafes, shops on OpenStreetMap. As you move around the map, an overlay highlights the POIs and colour-codes their accessibility for wheelchair users. Adding or changing the accessibility for a POI is as easy as clicking on it and clicking one of three options – job done! The data is then fed back to OpenStreetMap (so the whole community gets the benefit of this extra “tagging” information) and Wheelmap itself updates the colour.

The website is not perfect – the location finder is not yet fully localised from its German origin, for example, also the map controls aren’t styled like the rest of the site and the default OpenStreetMap map also shows its own symbols for many POIs, so they really need their own custom render – but this is nonetheless a great implementation targeting very specific data in the OpenStreetMap database and making it absolutely trivial for anyone to enhance the map in this way. I’m a firm believer that the easier it is to do something, the greater number of people will contribute.

Friday Review: London Crumpled City Map

I got the London Crumpled City Map as a Christmas pressie. It is a large scale map of central London – covering most, but not all, of Zone 1 – the eastern edge of the City is chopped off. It is designed and produced by an Italian company, and is one of a series of maps that also includes New York, Paris, Tokyo, Berlin and Rome.

The big feature about the map is you can scrunch it up into a ball and keep it in the soft pouch that comes with it – unraveling for later use is straightforward. The map is printed on a very thin and light plastic waterproof sheet, kind of like a synthetic tablecloth. The map remains very creased when it is flattened out – naively I was expecting it to spring back to its perfect condition which it was in when I opened the present – but is still very readable.

The cartography is simple and clear – grey roads with black text, on white. Large buildings are shown in lighter grey, parkland is in olive and the Thames is an unusually light blue. It’s too simple in places – paths in parks and on the Thames’s pedestrian bridges appear just like the roads, and bridges and tunnels are not shown, which means the Victoria Embankment appears to end abruptly as Blackfriars, rather than continuing under it as Upper Thames Street. Surface railway lines are present as narrow lines. Only the TfL stations are shown – City Thameslink is missing, for instance.

As a some-time contributor to the project, I’m pleased to see (thanks to the prominent credit on the map) that the data is from OpenStreetMap. Unfortunately the map does have a number of typos, more so than you would normally expect for a central London map. I’m not sure if these are due to the OpenStreetMap data – in which case the data must have been sourced a long time ago, as OpenStreetMap is pretty good in central London these days – or from an independent list of points of interest which have been superimposed on the top.

From the nature of the mistakes, I’m pretty sure OSM is not at fault here. For example, UCL appears as the University College of London, and Russel(l) Square station is missing an “l”. The Diana memorial fountain in Kensington Gardens appears as “Diana, Princess of Wales Memorial Fund”. A mysterious second “Paddington” station appears where the Bakerloo Line’s Edgware Road station is, and the other Edgware Road station doesn’t appear at all. Some of the landmarks are a bit dubious – the Royal Agricultural Hall is actually better known these days as the Business Design Centre in Islington and has been as such for at least 15 years. There are some prominent landmarks missing too, such as the Globe theatre on Bankside, the Old Bailey and the BFI IMAX cinema in Waterloo.

So – it’s probably not a particularly useful map for anyone – for tourists it leaves off too many attractions and stations, for cyclists and drivers it doesn’t distinguish between paths and roads, and for walkers, it doesn’t show the route of the underground lines. You would probably be better off looking at the many “minilith” map slabs that are starting to appear all around the city. However, it is very light, easy to store (there’s something very satisfying about crumpling it up) and completely waterproof. And it’s another real-world use of OSM. So I like it. Link to it on Amazon.

More Circles on a Map – Orienteering Fixtures

Five years ago, I created a mashup of forthcoming orienteering fixtures in Great Britain, as listed by the sport’s national governing body, British Orienteering, on its website. It was based on the Google Maps v2 API, and a regular scraping of the HTML on their website, and was a set of pins on a map, coloured by the number of weeks to the event. On clicking a pin, you got a popup balloon with details of the event, and a link to the organising club’s website. A postcode locator, based on data from the NPEMap project, was added, so you could focus on events in your local area. You could also filter out far away events.

A couple of years later, British Orienteering’s web developers added their own map to their website – Google Maps v2 API based, with pins coloured by the number of weeks to the event, and a popup balloon, a postcode search and distance filter etc etc… The Unique Selling Point of my fixtures map was lost.

So, when a rewrite of British Orienteering’s website just before Christmas broke my map, I took the opportunity to rewrite it, as a vacation project, using the technologies I’ve been using a lot in 2010 – namely OpenLayers, OpenStreetMap, OS OpenData and coloured vector circles. The map is bigger, brighter, and hopefully more useable than the official map and my previous version.

You can see the new map here – with a mass of dots representing forthcoming fixtures, and circles surrounding the “home” postcode, backed by OpenStreetMap, with the postcode locator based on CodePoint Open from Ordnance Survey OpenData. Only the locator uses a database, the rest of the webpage is constructed on-the-fly from a webpage regularly copied from the British Orienteering website.

Not Scarborough…

The map remains subject to the quality of the data entered on the corresponding list – there is some limited tidying up of the data, but it’s difficult to correct grid references that result in events being in the sea – there’s currently one in the Irish Sea, as the event registrant entered “GR” as the grid reference letters, and this just so happens to be the location of the GR myriad. There is still work to be done on my new map, such as spotting obvious errors like this, guessing locations where a grid reference isn’t supplied, and perhaps including Northern Ireland’s events.

Incidentally, my original orienteering web map, which inspired my fixtures map, was one showing orienteering maps, it was written way back in August 2004, using a Flash mapping package by Map Bureau, with dots superimposed on top of a map pinched from Wikipedia. We’ve come a long way.