Category Archives: OpenStreetMap

OpenStreetMap: London Building Coverage

osm_londondetail

OpenStreetMap is still surprisingly incomplete when it comes to showing buildings for the London area, this is a real contrast to other places (e.g. Birmingham, New York City, Paris) when it comes to completeness of buildings, this is despite some good datasets (e.g Ordnance Survey OpenMap Local) including building outlines. It’s one reason why I used Ordnance Survey data (the Vector Map District product) rather than OpenStreetMap data for my North/South print.

The map below (click to view a larger version with readable labels and crisper detail, you may need to click it twice if your browser resizes it), and the extract above, show OpenStreetMap buildings in white, overlaid on OS OpenMap Local buildings, from the recent (March 2015) release, in red. The Greater London boundary is in blue. I’ve included the Multipolygon buildings (stored as relations in the OSM database), extracting them direct from OpenStreetMap using Overpass Turbo. The rest of the OSM buildings come via the QGIS OpenStreetMap plugin. The labels also come from OS OpenMap Local, which slightly concerningly for our National Mapping Agency, misspells Hampstead.

The spotty nature of the OSM coverage reveals individual contributions. For example, Swanley in the far south east of the map is comprehensively mapped, thanks presumably due to an enthusiastic local. West Clapham is also well mapped (it looks like a small-area bulk import here from OpenMap) but east Clapham is looking sparse. Sometimes, OpenStreetMap is better – often, the detail of the buildings that are mapped exceeds OpenMap’s. There are also a few cases where OSM correctly doesn’t map buildings which have been recently knocked down but the destruction hasn’t made it through to OpenMap yet, which typically can have a lag of a year. For example, the Heygate Estate in Elephant & Castle is now gone.

The relative lack of completeness of building data in OpenStreetMap, for London, where the project began in 2004, is – in fact – likely due to it being where the project began. London has always an active community, and it drew many of the capital’s roads and quite a few key buildings, long before most other cities were nearly as complete. As a result, when the Bing aerial imagery and official open datasets of building outlines became more recently available, mainly around 2010, there was a reluctance to use these newer tools to go over areas that had already been mapped. Bulk importing such data is a no-no if it means disturbing someone’s prior manual work, and updating and correcting an already mapped area (where the roads, at least, are drawn) is a lot less glamorous than adding in features to a blank canvas. As a result, London is only slowly gaining its buildings on OSM while other cities jumped ahead. Its size doesn’t help either – the city is a low density city and it has huge expanses of low, not particularly glamorous buildings.

An couple of OpenStreetMap indoor tracing parties might be all that’s needed to fix this and get London into shape. Then the OpenStreetMap jigsaw will look even more awesome.

osm_london_2mb

Click for a larger version. Data Copyright OpenStreetMap contributors (ODbL) and Crown Copyright and Database Right Ordnance Survey (OGL).

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Tube Line Closure Map

anim

[Updated] The Tube Line Closure Map accesses Transport for London’s REST API for line disruption information (both live and planned) and uses the information there to animate a geographical vector map of the network, showing closed sections as lines flashing dots, with solid lines for unaffected parts. The idea is similar to TfL’s official disruption map, however the official one just colours in the disrupted links while greying out the working lines (or vice versa) which I think is less intuitive. My solution preserves the familiar line colours for both working and closed sections.

My inspiration was the New York City MTA’s Weekender disruptions map, because this also blinks things to alert the viewer to problems – in this case it blinks stations which are specially closed. Conversely the MTA’s Weekender maps is actually a Beck-style (or actually Vignelli) schematic whereas the regular MTA map is pseudo-geographical. I’ve gone the other way, my idea being that using a geographical map rather than an abstract schematic allows people to see walking routes and other alternatives, if their regular line is closed.

Technical details: I extended my OpenStreetMap-based network map, breaking it up so that every link between stations is treated separately, this allows the links to be referenced using the official station codes. Sequences of codes are supplied by the TfL API to indicate closed sections, and by comparing these sequences with the link codes, I can create a map that dynamically changes its look with the supplied data. The distruption data is pulled in via JQuery AJAX, and OpenLayers 3 is used to restyle the lines appropriately.

Unfortunately TfL’s feed doesn’t include station closure information – or rather, it does, but is not granular enough (i.e. it’s not on a line-by-line basis) or incorrect (Tufnell Park is shown only as “Part Closed” in the API, whereas it is properly closed for the next few months) – so I’m only showing line closures, not station closures. (I am now showing these, by doing free-text search in the description field for “is closed” and “be closed”.) One other interesting benefit of the map is it allows me to see that there are quite a lot of mistakes in TfL’s own feed – generally the map shows sections open that they are reporting as closed. There’s also a few quirks, e.g. the Waterloo & City Line is always shown as disrupted on Sundays (it has no Sunday service anyway) whereas the “Rominster” Line in the far eastern part of the network, which also has no Sunday service, is always shown as available. [Update – another quirk is the Goblin Line closure is not included, so I’ve had to add that in manually.]

Try it out

Visit the new oobrien.com Shop
High quality lithographic prints of London data, designed by Oliver O'Brien

Street Trees of Southwark

southwarktrees_rotherhithe
Above is an excerpt of a large, coloured-dot based graphic showing the locations of street trees in Rotherhithe, part of the London Borough of Southwark in London, as released by them to the OpenStreetMap database back in 2010. You can download the full version (12MB PDF). Street trees are trees on public land managed by LB Southwark, and generally include lines of trees on the pavements of residential streets, as well as in council housing estates and public parks. By mapping just the trees, the street network and park locations are revealed, due to their linear pattern or clumping of many types of trees in a small area, respectively. Trees of the same genus have the same colour, on this graphic.

southwarktrees_thinWhy did I choose Southwark for this graphic? Well, it was at the time (and still is) the only London borough that had donated its street tree data in this way. It is also quite a green borough, with a high density of street trees, second only to Islington (which ironically has the smallest proportion of green space of any London borough). There are street tree databases for all the boroughs, but the data generally has some commercial value, and can also be quite sensitive (tree location data can useful for building planning and design, and the exact locations of trees can also be important for neighbourly disputes and other damage claims. It would of course be lovely to have a map of the whole of London – one exists, although it is not freely available. There are street tree maps of other cities, including this very pretty one of New York City by Jill Hubley. There’s also a not-so-nice but still worthy one for Washington DC.

Also well as a PDF version, you can download a zip-file containing a three files: a GeoJSON-format file of the 56000-odd street trees with their species and some other metadata, a QGIS style file for linking the species to the colours, and a QGIS project file if you just want to load it up straight away. You may alternatively prefer to get the data directly from OpenStreetMap itself, using a mechanism like Overpass Turbo.

A version of this map appears in London: The Information Capital, by James Cheshire and Oliver Urberti (who added an attractive colour key using the leaf shapes of each tree genus). You can see most of it below. I previously talked about another contribution I made to the same book, OpenStreetMappers of London, where I also detailed the process and released the data, so think of this post as a continuation of a very small series where I make available the data from my contributions to the book.

The data is Copyright OpenStreetMap contributors, 2015, under the Open Database Licence, and the origin of most of the data is a bulk-import supplied by Southwark Council. This data is dated from 2010. There are also some trees that were added manually before, and have been added manually since, by other OpenStreetMap contributors. These likely include some private trees (i.e. ones which are not “street” trees or otherwise appear on private land.) Many of these, and some of the council-data trees, don’t have information their genus/species, so appear as “Other” on the map – orange in the above extract.

southwarktrees_book

Primary Roads

a1a6_london

Britain’s “top” primary roads – the A1, A2, A3… to A9 – are arranged in a particular pattern, with the A1-A6 radiating out clockwise from London and the A7 to A9 similarly radiating around Edinburgh.

I used Gemma, an old UCL CASA project that Steve and I worked on back in 2011, to draw, from OpenStreetMap, the routes of the A1-A6 as they leave London. The A5 has a gap between Edgware and Harpenden, and the A6 only starts at Luton – both of these changes likely due to the building of the M1 motorway which effectively replaced those sections. Co-numbered roads are not included in the map due to a conflict with the way OpenStreetMap and Gemma separate information. Key for the maps: Red = A1, Orange = A2, Green = A3, Blue = A4, Purple = A5, Black = A6.

Also of interest is that the only two roads that “touch” in London are the A2 and A3, at Borough. The other roads may at one time have converged at junctions, but their starts have been shortened slightly over the years. The big junction at Bank certainly looks like a place where the A1, A3 and A4 could have started from. (Outside of London, the A7 touches the A1 at its northern end and the A6 at its southern end.) Diamond Geezer walked the first mile of the A1-A5 a few years ago.

Gemma still partially works, despite not having seen much love for the last few years and having never made it out of beta (it was a short project). It is recommended you use the OpenStreetMap (or Marker) layers only, to avoid bugs, and watch out if removing layers. You can see the live A1-A6 map here or have a go at building your own.

a1a6_detail

Key for the maps: Red = A1, Orange = A2, Green = A3, Blue = A4, Purple = A5, Black = A6.

I’ve blogged about Gemma before (more).

The coloured road lines are Copyright OpenStreetMap contributors and the greyscale background map is Copyright Google.

OpenStreetMappers of London

IMG_1370

I contributed a number of graphics to LONDON: The Information Capital, a book co-written by Dr James Cheshire, also of UCL Geography. Two of my graphics that made it into the book were based on data from OpenStreetMap, a huge dataset of spatial data throughout the world. One of the graphics, featured in this post, forms one of the chapter intro pages, and colours all the roads, streets and paths in the Greater London Authority area (around 160,000 “ways” which are discrete sections of road/path) according to the person who most recently updated them. Over 1500 indivdual users helped create and refine the map, and all are featured here. I was pleased to discover I was the 21st most prolific, with 1695 ways most recently modified by myself at the time that the graphic was produced.

The more active users will typically have areas around home and work which they intensively map, plus other, smaller areas such as contributions made during a mapping party or other social event organised by/for the London OSM community. Here’s an example filtering for just one user:

osm_dan

Putting the users together reveals a patchwork of key authors and more minor contributors, together forming a comprehensive map of the city. Detail levels vary, partly as the fabric of the city varies from area to area, but also as some contributors will be careful to map every path and alleyway, while others will concentrate on the driveable road network.

osm_detail

The data was obtained from a local copy of the OpenStreetMap database, for Great Britain, that I maintain for various pieces of work including OpenOrienteeringMap. You can obtain the data files from GeoFabrik (this link is to their new London-only version). The data was captured in early February 2014. Newham borough in east London (light blue) shows up particularly prominently because it looks like it had had a bulk update of all roads there by a single user, just before the capture, to indicate which were lit by streetlights (lit=yes).

I used QGIS to assemble the data and applied the temp-c colour ramp, classifying across all the contributors – I then changed the ones which were assigned a white colour, to green. The colours used in the book are slightly different as some additional editing took place after I handed the graphic over. The colour ramp is relatively coarse, so multiple users will have the same colour assigned to them. The very long tail of OSM contributions (where only a small number of people make the great majority of edits) mean that this still means that most major contributors have a unique colour assigned to them.

osm_book
View larger version.

Download:

Note that these files actually are for an area that is slightly larger than the Greater London Authority extent – a buffer from Ordnance Survey Open Data Boundary-Line is used to mask out the non-GLA areas.

If you like this thing, it’s worth noting that Eric Fischer independently produced a similar graphic last year, for the whole world. (Interactive version).

Ironways of London

london_ironways

It’s always irked me slightly that many online maps of London show the various tube services as straight lines between stations, or as idealised Bezier curves. Perhaps the regimented lines and angles of the official “Beck-style” tube diagram has meant that, when translating into a “real life” geographical map, people have tended to keep the simplifications. After all, if you are travelling around London on the tube or railways, only the location of the stations are important – not how you travel between them.

Focusing on the section of the DLR just south of Canary Wharf:

Google Public Transit view, using Bezier curves between stations:
iw_google

A typical “straight lines between stations” map – from CASA’s own MapTube:
iw_maptube

One of DLR’s own official diagrammatic maps:
iw_official

Where the line actually goes:
iw_actual

OpenStreetMap contributors have faithfully mapped most of London’s railways, including best-guess alignments for tube tunnels, using ventilation shafts on the service and “feeling” corners and curves that tube trains take – bearing in mind that GPS does generally not work underground. There are a couple of minor mistakes, such as orientations of the Northern Line curves near Mornington Crescent, and a part of the Piccadilly Line in north London.

I’ve taken this now excellent dataset, and as part of work to produce a comprehensive vector file of Transport for London (TfL) service routes, I’ve produced this interim map – the Ironways of London. TfL’s public service routes are highlighted in green. Lines in red are other train operator routes, sidings and depots, freight rail routes, disused lines, unusual chords and the odd ornamental railway. Many of these are obscured by the green lines of TfL routes, where the two coincide. There are a few missing sections, e.g. a couple of tunnels to the south of London are not shown.

The map here uses Google aerial imagery as a background, Ordnance Survey Open Data to show the boundary of Greater London, and OpenStreetMap to show the rail routes themselves. As such, it’s a nice mashup of the three major sources of free-at-point-of-use spatial datasets for London.

Here is the full size version.

There are a few other examples around on the net of the same thing – here’s an ESRI one. The Carto Metro one is excellent and is a level of detail beyond what I am aiming for.

In the new year I hope to complete and release the tidied vector data. [Update: Data released, more info.]

Extracting Feature Geometries from OpenStreetMap

minnea_summary

I’ve recently been extracting some river geometries for major cities around the world. The data needs to be a list of latitude/longitude coordinates, representing the nodes on the shape for the river concerned.

I’m sure there’s easier ways to do this, but here’s my technique, shown here for Minneapolis. Click the images for larger version.

1. Extract the data from OpenStreetMap. Use the Export function, and draw out the area concerned with a bounding box. Choose OpenStreetMap XML as the format. I originally tried SVG, but this presents you with screen coordinates instead of latitude/longitude pairs.

2. Open the resulting file in Quantum GIS (QGIS). I used QGIS 1.9. You need the OpenStreetMap plugin installed, this will allow the OSM file that was created in Step 1 to be read straight in (in fact you could download the file directly from the OSM servers, if you wanted to).

3. Select the feature you are interested in. My river (actually a waterbank polygon) is a “hairy feature” as it extends well beyond the extent of the data that was downloaded. Make sure you are selecting it (feature turns yellow) rather than highlighting it for feature information (feature turns red). Otherwise, the subsequent file is, rather unhelpfully, blank.

4. Do Layer > Save Selection as Vector File. Choose “KML” as the format. You probably don’t need to change the coordinate reference system (CRS) as the data will already be in WGS 84, and this (“normal GPS-style latitude/longitude) is the CRS you want.

5. Edit the resulting file, removing the XML tags, and header/footer, and replace spaces with return characters, to leave a long list of latitude/longitudes, ready for importing into your visualisation code.

Run Every Street in Edinburgh – in Strict Alphabetical Order

…it sounds like one heck of a lot of running. But Murray Strain, one of Scotland’s top terrain runners, is counting on it for his basic training. He’s logging the whole venture, which is based on his trusty Edinburgh A-Z. If two adjacent streets with very similar names are nonetheless separated in the A-Z index by one on the far side of the city, it means a couple of legs right across the city.

Since he started the exercise last year Murray’s got through all the As, and is currently midway through the Bs. I’ve produce a couple of GEMMA maps, one showing the A-Bs (above, As are red and Bs are orange) and one showing the A-Gs (below, in rainbow order). That’s a lot of streets. N.B. The maps in fact show all linear features in the area in OpenStreetMap, so the odd named cycleway and waterway has crept in there too. But the ~95% of the coloured lines will be the streets that Murray will be run.

In order to produce the map, I’ve added a new feature to GEMMA – it now allows you specify only one desired geometry type, i.e. points, lines OR polygons, when adding an OpenStreetMap layer to your map. Previously, you got all three types, although you could reduce each to a dot if desired. This example also highlights the need for legends on the PDF maps that GEMMA produces – a larger coding change, so one that would make it into a future version 2 of GEMMA.

Main Street UK

GEMMA is the project I’ve been working on for the last six months, it’s one of the JISCgeo projects and it is now released – although consider it to be beta as there are lots of bugs and UI quirks that we are aware of. More about GEMMA can be found on the project’s blog.

One use of the OpenStreetMap feature highlighter in GEMMA, that was suggested by one of the participants at the JISCgeo Meeting earlier this week where we launched the web application, and augmented by a friend who was trying it out, was mapping the occurrences of the “High Street” road name – and a few regional variations, namely Main Street, Front Street, Market Street, Fore Street and The Street. Using GEMMA, and the high level of completion of OpenStreetMap in the UK and Ireland, allows us to visually show the spatial patterns of such street names.

Here’s a stitched-together screenshot of the GEMMA webpage showing the pattern throughout the UK and part of Ireland:

It turns out that Main Street is popular in the Midlands and in Scotland and Ireland, and Front Street is popular in the North-East of England (around Newcastle) while High Street is used nearly everywhere in the UK – but only sparingly in Ireland. Market Street is popular in the Manchester and Devon areas. Fore Street is popular in Cornwall and The Street very popular in Essex and Kent.

Note that many parts of Northern Ireland and the Republic of Ireland, are not yet well mapped in OpenStreetMap, so the street names will be missing in some parts here. The base-map is copyright Google and the street data is CC-By-SA OpenStreetMap.

You can see the live version of the map here.

Olympic Torch Relay – The Unofficial Map

The organisers of next year’s Olympic Games in London, LOCOG, have unveiled their map of the 1000+ places that the Olympic Torch Relay will pass through. The data that the map is built from is readily accessible (as a JSON file that gets downloaded to your computer when you view the map) so I’ve taken the data and built my own (unofficial) map. It has a number of advantages over the official map:

  • The base map is OpenStreetMap, which is much more detailed.
  • The map takes up the whole browser page, allowing for easier panning around.
  • The line that connects each of the places is drawn as a vector, so it still appears as you to zoom right in to see individual villages. (The official map surprisingly uses tiles for the line.)
  • There are Wikipedia links for each of the places. Almost all of these resolve to proper Wikipedia entries, so you can easily find out about the places that have been picked, with the richness of detail that is characteristic of the Wikipedia project.

The route has been designed to “ensure the flame within a one-hour journey of 95% of people in the UK” (Source).

See it here.